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ABSTRACT 

In the last several years, one of the approaches that 
has been advocated as a mechanism to bring about needed Improvenents 
and revitalize education is the use of incentives. This paper 
suggests a conceptual framework for studying the uses of incentives 
in American public education and examines briefly the 
state-of-the-art of recent research and practice incentives. The 
primary focus of the paper is on incentives as they reliate to the 
organization and governance of education in the public school 
systems* Several incentive programs are examined on the basia of 
three important characteristicsi the reward offered, the individuals 
or organizations that may receive the reward, and program goals 
and/or results required to obtain the reward. Following consideration 
of these programs, a basic concept underlying all incentive programs 
is formulated. A conceptual framework is then proposed that has three 
main variables; (1) program targets, (2) goals/results, and (3) 
reward. Conclusions and recommendations are based not only on the 
material presented in this paper, but also on the literature 
reviewed. A 40-item bibliography is included. (Author/JP) 
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A CnNCKPniAL FRAMEWORK FOR TIIE STUDY OF INCEmTVES IN 
AMERICAN PUBLIC EDUCATION 

I. imTODUCTION 

At Purposes and Limitations 

In the last several years one of the approaches that has been 
advocated as a mechanism to bring about needed improvements and revitalize 
education is the use of incentives. Incentives, it has been argued, may be 
an essential ingredient in upgrading teaching tecliniques or in ijnproving 
student performance. A wide variety of experimental and operational incentive 
programs has been tried and still others proposed. 

The purpose of this paper are to suggest a conceptual f ramewrk 
for studying the uses of incentives in American public education and to 
examine briefly the state-of-tlie-art of recent research and practice on 
incentives. Review of the literature has been confined largely, although 
not exclusively, to the last 5 years. It is not, howcY«, the purpose of 
this paper to present, per so, a review/ of the literature, but rather to 
use available infomation in developing concepts and providing illustrations. 

The prijrary focus of this paper is on incentives as they relate 
to the ovganization and governance of education in the public school systems 
of the United States. Consequently, the extensive literature dealing with 
incentives as used in classroom learning studies and experiments is only 
treated peripherally. 

B. Approach 

In this paper the term ^'incentive program*' is used to refer to a 
plan or program which by self -definition uses incentives as specific COTiponents 
of the prograiii or, in some cases, meets one or more of the meanings of 
incentives included in the definitions found in Appendix A. 



Several incentive programs are examined on the basis of three 
important characteristics: the reward offered, the individuals or organiza- 
tions that may receive the re^vard, and program goals and/or results required 
to obtain the reward. Following consideration of these programs, a basic 
concept underlying all incentive programs is formulated. A conceptual 
framework is then proposed. Tlie framework has three main variables: 
(1) program targets, (2) goals/ results, and (3) reward. 

Conclusions and reconjnwidatibnli l^r^^ased not only on the 
material presented in this paper, but also on the literature revie\\'ed as a 
part of the study which is then presented. V 

Appendix A to the paper contains definitic|ns of incentive as 
used in several of the social science disciplines. Api^^ndix C presents 
descriptions of several incentive programs in American education; the 
descriptions were derived from the literature. 
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II. CONSIDERATIONS IN THE DEVELOPMENT OF A CONCEimL FRAJtEh^ 

A wide variety of proposed, experimental, and operational programs 
in public education have been labeled as incentive programs. These applications 
have ranged from performance contracting (called incentive contracts by some, 
e.g., Stucker and Hall (38)) to merit pay for teachers and including the use 
of incentives in the classroom for students to enhance learning, the latter 
sometimes in conjunction with bonuses for teachers and parents. Appendix B 
of this report describes seven examples of programs which use incentives as a 
basic component. These examples are neither representative nor exhaustive. 
They were chosen to illustrate not only the wide variety of incentive programs 
but also some of the ideas advanced in this paper. 

Considering the wide variety of incentive programs, it seems 
reasonable to ask whether there are any common characteristics or conponents 
of the programs which may be abstracted to serve as the basis of a conceptual 
framework. This section addresses this question. 

It is the position taken in this paper that incentive programs 
have three aspects which may be abstracted to serve as a basis for developing 
a conceptual framewrk. Those three aspects are: 

(1) A reward is offered. 

(2) There are goals or results to be obtained. 

(3) Individuals or organizations may obtain the reward by 
achieving the goals or results. 

In this section several incentive programs will be examined from 
the three points of view indicated by these aspects. 
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A. Targets of Incentive Programs 

The targets of incentive programs are the entities Wliich may 
receive the reward. Thus, in a program in which teachers aro to be paid a 
bonus dependent on student learning, the teachers are the target, even 
though the students benefit as a result of better learning. Incentive 
programs may have provision for either individuals or organizations as 
targets. The incentive programs which have received the most attention 
are those which liave been directed toi>rard individuals; that is, personnel 
in the school system. 

Coleman, for example, has noted that incentives in e<lucation 
are "...merely a special case of the general point that any organization is 
a system of interdependent incentives, and the functioning of the organiza- 
tion depends upon the adjustment of these incentives.*' (7, p. 1) He goes 
on to say that incentives may be adjusted for: 

(1) The School Superintendent 

(2) Staff members in the Administrative Office 

(3) Principals 

(4) Teachers 

(5) Children 

(6) Parents. 

Jung, Lipe, and Wolfe identify the same general target groups 
(21, p. 20) and go on to point out that incentives may be provided to these 
target populations either individually or as groups (e-g., all or none of the 
teachers in a particular dcpartinent, school, or district). Obviously, incentive 
programs may be directed at various combinations of these target groups. 



Programs which have had various categories and corrsbinations of 
individuals as targets have included: many experimental programs which used 
incentives to improve academic performance; two major programs - one testing 
of 0B3 and one by USOE - of large scale incentives in education; programs 
that have attempted to introduce merit pay for teachers using academic 
performance by students as the basis of the reward; as well as a number of 
operational merit pay plans, most of which do not use student achievement 
tests as measures. Most of the experimental programs have had students as 
the targets. Lipe and Jung (23) provide a revic\>r of the literature in this 
area which is substantially the same as that provided by Jung, Lipe, and 
IVblfe. (21) The latter also review in some detail eight student incentive 
programs. (21, Appendix A) 

Some experimental programs either focused on the teacher or 
included the teacher and sometimes the parent as a part of a more inclusive 
design. Fox and Jung (12), for example, report a study which combined 
student, teacher, and some parent incentive in a brief experiment. They 
suggest that the results, although quite tentative, generally support the 
notion that incentives tend to enhance school learning. Tlie nature of 
design, including the three types of incentive, does not yield information 
on which category of incentive may be most effective. 

OEO, as a part of the largo- scale study relative to performance 
contracting. Included an effort (29) to test the effects of "incentive only." 
This program involved contracting with local school districts, which in turn 
contracted with t!ie local teacher association to provide incentives to 
students and teachers. The OEO report includes results that indicate the 
use of incentives did not seem to contribute to academic (math and reading) 
improvement on the part of the students. 
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During the school year 1971-72 the U. S. Of f ice of Education 
designed and implemented a 1-year project on the use of incentives in education. 
The experiment involved four school districts. In each district there were 
experimental and control schools. Two of the districts had "Incentives to 
teachers only'' programs, and two of the districts had 'Incentives to teachers 
and parents programs. 

Planar Corp. (31) in its impact evaluation report generally 
concluded that the teacher only model did not demonstrate any differentially 
positive effects over the control school in improving student performance. 
On the other hand, the teacher/parent model did seem to have such effects. 
The report notes, however, that two features of the design impose severe 
limitations on the inferences which may be drawn from the study, (31, IV - 7fi8) 
These design features are the lack of randomization in assignment of treatments 
to units and the small number of experimental units (i.e., only two per 
condition) . 

The OBO performance contracting study, of which the incentives 
only experiment mentioned above ^^as a part, has also been criticized for lack 
of randomization in key features of the design. (28) 

Merit pay programs have raised considerable controversy, usually 
because they have emphasized student academic achievement as the basis of the 
reward. 

Thomas and ^fcKinney, who treat incentive or merit pay plans as one 
of several possible programs that are part of the accountability movement, 
point out that: 

The incentive pay plans advocated today 
seek to replace or supplement a salary 
schedule based on education and longevity 
with a plan tliat provides rewards to 
teachdrs commensurate \dtn performance. 
O (43, p. 40) 
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The usvial incentive is a salary differential generally awarded 
for superior performance in the standard teacher role. In this context 
superior performance is intended to be determined by measures of student 
achievement or perhaps classroom observation. 

The most important variation in merit or incentive pay plans 
according to Thomas and ^fcKinney is differentiated staffing • 

Differentiated staffing generally provides not only for differ- 
ential roles for teachers but also for differential pay according to the 
role. (48) Within any one role, however, there is usually not a salary 
differential based on student achievement, which is, as noted above, 
supposed to be a prime factor of merit pay plans. 

An article in Na tions Schools (49) discusses briefly four merit 
pay plans including the so-called Clark plan for Washington, I). C, which 
proposed a four-level hierarchical staff of teachers with promotion from 
one level to the next higher level dependent in large part on student 
advancement in math and reading as measured by standardized achievOTont 
tests. This plan ivas opposed, it would appear successfully, by the teachers' 
union. 

Rhodes a|id Kaplan (32) include dexcriptions of 11 merit pay plans, 
none of which seems t6 have as a feature tlie measurement of student achievement 
a prime or even partial indicator for determining re\\^rd. 

One salary plan for teachers described by Chaplin (5) as a merit 
plan seems to be essentially a salary schedule of the more traditional prepa- 
ration-longevity type. Although targets for incentive programs may include a 
number of personnel in the educational system, when the experimental classrocMu 
studies are excluded, almost without exception the target has been teachers. 

o 
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Two exceptions to this may be noted. Uke County, PloriUa has a plan which 
as described in "Oiitlook for Teacher Incentives'' (49) uses a salar)^ schedule 
based on training, experience, and school size plus a bonu s of $980.00 for 
principals . Tlie bonus is conditional upon undertaking and completin^j an 
approved project. The other exception is the differentiated staffing plan 
being implemented in Temple City, California which redefines the traditional 
principal's role, eliminating the usual administrative details and emphasizing 
skills in grouj) dynamics and social leadership. (37) 

For incentive programs have had personnel as targets, by 

far the greatest number has been directed at students. These are not, how- 
ever, germane to this paper. Of the balance, the bulk of the programs have 
had teachers as the targets. In some experimental programs teachers, in 
combination with students and/or parents, have been the targets. In some of 
these exi^erimental programs the reward has been given to all (e.g., all 
students in the classraom) only when the group achieved some minimum or 
average level of per forifvuice; othendse, no one received the rerard. In all 
cases, however, the net effect has been that an individual benefited. 

All of the foregoing programs have, therefore, had one or more 
sets of individuals as the target. Another u'liole class of programs includes 
those directed at an organizational entity - schools or school districts, 
for example. One type of program aimed at school districts is intended to 
alter the form of organization itself. IXiring the last half century one of 
the prijr.e movements in .American public education has been the movc^nont toward 
more and larger compreliensively-organized school districts, which offer K-12 
or 1-12 grades under a central administration. In California such conpre- 
hensively organized districts are called unified districts- The State of 
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California, as provided for in the California Mication Code of 1059 as 
amended through 1972 (45), intends to have unified districts as the primar)' 
form of school district organization. To encourage the formation of such 
districts, additioail monies over and above the basic state aid are offered 
to unified districts that meet certain conditions. This aid is (based on 
the 1969 level of support) alwut 2.6 percent and 2.1 percent of the basic 
aid for elementary and secondary school pupils, respectively. 

T\%-o of the reasons that have often been advanced for consolidating 
school districts are: (1) better educational, more d ivers if ictl programs and 
(2) savings, for example, through elimination of dunlicate facilities. One 
approach has been suggested to address directly the question of savings as 
a result of proper organization. 

Colin and Millman (6, p. 88) state that "considerable evidence 
demonstrating the existence of substantial scale economies in public 
(especially secondary) scliools has been presented in recent years 
Tliey suggest t)iat . proper size of school districts may result in economies 
of scale, and they propose an approach. Unlike the California program, vv-hich 
is concerned with the form of school organization, Cohn and ^?illman are 
concerned with the (luestion of size of the organization. 

Another group of incentive programs which are aimed at t1\e 
■organizational level is not intended to change the organization per se, but 
rather hope to alter the structure or operation of education. 

Johns (20) proposed to modify tlie Florida State Aid Program, 

a foundation program, by providing a portion of state monies by comparing 

the ratio of: (1) local (district) actual effort to support education in 

relation to the local effort required for a minimum program with (2) state 

average ratio of actual effort to minimum effort. This proposal to affect 

O "Istribution of state aid monies differs from the Cohn and Millinap proposal. 
ERIC 
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The latter were directly concerned with affecting organization, whereas hero 
the concern is with structure; that is, the mechanism or operation of local 
(district) support of education, 

Jolms makes clear that an incentive program such as he proposes 
must be developed in the context of a particular state and its aid program 
and mentions that New York, Rhode Island, and U'isconsin have incentive programs. 

Sane incentive programs have had as an objective the changing of 
the structure or operation within the educational system. One example of this 
t>T3e of program is Title I of the Elmentary and Secondary Fducation Act, 
which provides Federal assistance to local education agencies to improve 
educational programs for children of low- income families". • .to meet the 
special educational needs of educationally deprived children. (9) The 
incentive portion of this program is in Part B, which provides for an 
incentive grant for qualified states that apply. Qualified states are those 
whose index of effort (the ratio of all non-Federal monies spent on education 
to total personal income) exceeds the national index (average of all states). 

In sunwary, incentive programs may have as targets either 
individuals or organization entities. Incentive programs which have individuals 
as targets may be further described by the classes of personnel who are the 
targets within the school system. Incentive programs aimed at the organiza- 
tional level, in contract, may be further r>ubdivided by the nature of the 
change that is sought: changes, on the one hand, to the organizational 
arrangement or changes, on the other hand, to the structure or operation. 
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®' Coals/Results of Incentive Pro;;rains 

Three characteristics of incentive programs were identified and 
need to be examined in developing a conceptual fraine\\ork. These cliaracter- 
istics are the: 

(1) Entities that may obtain the reward (as discussed above, 
the targets) 

(2) Results to be obtained 

(3) Notion of a re^vard. 

The purpose of tliis section is to discuss the second of these 
characteristics. 

One analytic teclinique for examining education is to view educa- 
tion as a system which has certain inputs, uses certain processes, and produces 
certain outputs. Tliomas and McKinney C^^^), for example, in discussing the 
testing and evaluation issue in the "accountability movement,'^ use this 
approach to distinguish- the three parts of the system that may be evaluated; 
these are: (1) inputs (e.g., funding), (2) processes (e.g., teacher performance), 
and (3) outputs (e.g., student learning). This system's analytic approach 
will be used to examine the goals/ results aspect of incentive programs. 

The notion that the reward will be obtained when the results are 
achieved implies three additional ideas. First, that goals and results are 
consistent; that is, if the program goal is to alter an input, then the 
results (the conditions to be achieved to obtain the reward) will also be at 
the input level. Second, a procedure exists (method, process, technique) by 
wJiich the results may be obtained. Third, there is a method for ascertaining 
whether the results have been achieved; that is, there is a method of measure- 
ment. Thus, the goals/results aspect of incentive programs may be examined 
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from three points: (1) goals/rosults consistency, (2) procedures to be 
emjUoyed, a!\d (3) the measurement method, 

1, Output Domain 
' , ' Those: Incentive programs which use standardized achieve- 
ment tests to measure, for example, student gains in reading and/or math are 
obviously dealing with results in the outpu.-. domain. The USOE incentive project 
evaluated by Planar Corp. (31) and Blaschlce (1) was of this type. This program 
had as a goal the demonstration of the feasibility of offering incentives to 
improve school learning, vliich is clearly in the output domain. The specific 
results were also in the output domain. Bonuses were to be paid to teachers 
only or to teachers and parents (the two incentive models) depending on the 
extent to which students showed Improvement in fractional grade equivalencies 
(GE) over a basic gain indicator (BGI) for reading and math separately. The 
following table showy the required results and the associated payments 



Improvement 


Teachers 


Parents 


Less than BGI 


None 


None 


BGI 


$150 


$12.50 


BGI + .1 GE 


300 


25.00 


BGI + .2 GE 


450 


37.50 


BGI + .3 GE 


600. 


50.00 



The instrument used to measure student gain was the 
Ifetropolitan Achievement Test Batteries! with average (mean) class improve- 
ment used to determine the amount of payment. 

In this program the methods to be used by the teachers 
to improve student learning were not specified. Teachers in the experimental 
schools were free to select those nethods or combinations of methods which 
they thought wjuld best produce the desired result. It must nevertheless be 
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taken as a given that It was a basic assumption of this program that such 
methods do in fact exist, and that such method (s) were probably different 
than those used in the control schools. 

The OBO study of performance contracting v-as fimilar, 
in the respects discussed above, to the USOE program. The OH) study: (1) had 
as a goal the improvement of student learning (output) , (2) specified output 
results for which reAs-ards wuld be paid, (3) Used standardized achievement 
tests as the measurancnt method, and (4) left the choice of method up to the 
contractors or, in the case of the incentives only portion of the study, to 
the teachers. This exclusion of methodology as an explicit consideration in 
the ODD study has been criticized by Miller (28) as a serious omission. It 
has been argued on the other hand, however, that if contractors (or teachers) 
are to be accountable (or paid) on the basis of output, then it is improper 
to control the methods that may be used to obtain the results. (40) 

■ " * ■ 

2: Input Domain 

In contrast to the foregoing examples which dealt with 
educational system outputs are those incentive programs which have as a general 
goal the alteration of an input characteristic. Those programs cited earlier 
as having organizational entities as the target have the change of an input 
characteristic as the goal. The State of California program (45) to achieve 
unified school districts as the basic form of school organization is of this 
type (assuming, of course, that one is willing to define the form of school 
organization as being in the input domain). In this case the result and the 
goal are identical; that is, the reward is obtained only by unifjod school 
•districts meeting certain conditions. The measurement method is essentially 
an administrative process determined in part by the provisions of the California 
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Wucation Code and in part by niles and regulations of the State Board of 
Education, and carried out by tho Superintendent of Public Instruction. In 
this case tho method by which unified districts are to be achieved is a 
political procedure and is spelled out in the code, (45, Division 5, 

Cohn and Ktillmn v.'ho propose an approach to obtain 
economies of scale also expect to affect school organization. Tliey state: 
"An explicit incentive structure in the state aid process would not only 
provide a certain degree of stimulus to change but would also serve to focus 
attention on tho scale issue." (6, p. 89) The Cohn andMillman approach 
differs from the California program in four important ways. First, it is a 
proposed rather than an operational program. Second, it is limited to the 
secondary school level. Third, it is the size of the organization rather 
than its form that is of concern. Fourth, it requires the assun^tion that a 
proper or optimum- sized school (or district) may be determined, In discussing 
this last issue they (6) note that ..most of the studies indicate a U-shaped 
relationship between per pupil cost and school size, measured by enrollment. 
It follows that most schools are either too large or too small...." (p. 89) 
Their report goes on to note, however, that there is some indication that 
U-shaped relationships may not be accurate and that it is possible that 
costs might decrease indefinitely as school size increases, (p. 91) Never- 
theless, they assume the U-shaped relationship in proposing a program of 
promoting economies of scale. 

Cohn and Mil Iman propose, for example, that as districts 
deviate from the optimum size they bo penalized an appropriate amount of 
their state aid monies, and the greater the deviation, the greater the 
penalty. In this example the purpose of the proposal may be taken as "to 
obtain optimum- sized school districts" and the results as "the closer to 
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optimum, the smaller th' nalty." Thus, both the goal and the results are 
in tho input donviln. 

l1io measurement method required for this program, while 
tcclmically complex to establish, is relatively simple to operate onco in place. 
It involves the determination of an optimal sized school (district), the unit 
cost of sucl) a district, the unit costs for each district in the state, a 
lcgislatively-dstablis]\ed scale factor, and the use of enrollment data (ADA) 
to compute the appropriate penalty factors and the loss of state aid monies. 
Colin and Millman do not specify procedures by which schools or districts 
could reduce their deviation from optimum size, although in most cases it is 
1 ikely that pol it ical procedures would be required . 

Johns' proposal (20) like Cohn and Millman also deals 
with the distribution of state aid monies. Unlike Cohn and ?fillman, however, 
Johns is not concerned with the organizational question but rather with 
structure, in this case with the mechanisms by which the amount of local 
support for education is detewiined. Johns states the goal of his proposal 
as follows: 

The purpose of this incentive grant is 
to give additional state financial help 
to those districts that are willing to 
help themselves financially. (20, p. 9) 

Districts would receive additional state aid money if 
the ratio of their local effort to local requirement exceeded a state average 
ratio of effort to requirement, whereas districts whose ratio fell below the 
average wuld lose a portion of the state aid money. Thus, both the goal 
and the results (the conditions for which the reward would be paid) are in 
terms of funding and clearly in the input domain. The measurement method 
is essentially administrative and involves the computation of the appropriate 

o 
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Indicies and ratios from data which would bo supplied by the districts in 
routine reports. JohjAS addresses the changes required in the Florida laws 
governing tho state foundation aid program but does not deal with tho ; 
question of how districts may go about altering the amount of local support. 
It may be assimed, however, that the procedures are those that determine how 
local school taxes are levied and are political in nature. 

Incentive programs with individuals as targets may have 
goals/results in the input dcsnain. Programs which deal with teacher preparation > 
including continuing professional groArth, may be of this type. Chaplin (5) 
reports a program in the Ilartford, Wise. Union High School District/ which 
has as its goal the continuing professional gro^rth of teachers and where 
professional gro^rth is essentially synonymous with additional academic 
preparation. The plan involves a pre salary schedule 

which has seven levels of academic preparation, from R\ toMV + 18 units, 
and for each level an ajllowancc for longevity from 1 annual increment at the 
BA level to 10 annual increments at the M/\ + 18 level. This program has both 
its goal and its results in the input domain. The measurement method of this 
program is practically self-administering and clearly involves no more than 
evidence of successful completion of academic course work, as well as records 
for longevity. The procedure is also quite clear * one merely enrolls in 
and passes academic course work. 

3. Process Domain 

Although they are generally less conwon, incentive 
programs whose goals/results are in the process domain of tho educational 
system may also be found, Lipe, Weisgerber, and Vox (24) report a project 
to develop and validate a package to conmunicate the use of incentives to 



17. 

olemontary and TnidcUe school teachers and administrator?. Stucker and Hall 
(39) describe a performance contract in which the Institute for Development 
of Education Activities undertook to teach individualized instructional 
techniques to teachers. The former, however, was not developed as an incentive 
programv and In the latter the incentives were to be paid to the contractor 
rather than the teachers. Both programs, however, clearly deal with the 
process domain of the educational system. An example of a program which has 
goals/results in both the input and process domains is taken from Rhodes and 
Kaplan. (32) The goal of the program is, as Rhodes and Kaplan quote from the 
district handbook, to "recognize the quality of teacher porfoimance as well 
as the quality of teacher training and experience." That portion of the goal 
dealing with training and experience is the input domain, and the portion 
dealing with performance is the process domain. This program uses three 
similarly-structured preparation/longevity salary schedules. Each schedule 
provides for the same levels of preparation and increments of experience. 
The differences, however, are the amount of pay for corresponding steps 
and the amount of the annual salary increment . Teachers may , with the 
specified years of experience, bo considered for promotion to the higher 
schedules. Promotion as well as retention on the higher schedules is 
determined by three factors: (1) a valid certificate, (2) professional 
growth (6 hours of graduate work) , and (3) an appropriate evaluation rating. 
The certificate and professional growth requirements are in the input domain. 

The measurement for evaluation is basically an observational method. 

Evaluation is done by the administrative and su\iervisory 

staff for all teachers as they become eligible and every 2 years thereafter. 
Written evaluation forms arc used and reviewed with the teachers, who have 
the right of appeal. 
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Teachers arc evaluated on five factors; (1) personal 
fitness for teaching, (2) classroom effectiveness, (3) relationship with 
students, (4) relationship with parents, and (S) relationship with staff . 
Each factor is rated on a five-point scale: (1) outstanding, (2) excellent, 
(3) above satisfactory, (4) satisfactory, and (5) needs Mprovonent. 
Evaluation is done in accordance with a witten guide, with most weight 
given to classroom effectiveness. 

It is expected tliat teachers on the Basic Schedule 
will generally receive ratings of satisfactory. Teachers on the Career 
Schedule must have ratings of above satisfactory to excellent. Finally, 
teachers on the Master Schedule must have ratings of excellent to outstand- 
ing. 

An example of a program with a goal in the process 
domain but targeted at an organizational entity is the Title I program of 
the Elementary and Secondary Act, Part B of which provides for special 
incentive grants. Tlic goal of the program and the use of the incentive 
funds are quite clear. 

Sec. 101 of the act states (in part): 

...The Congress hereby declares it to be 
the policy of the United States to provide 
financial assistance (to local educational 
agencies) * to expand and improve their 
educational program ... to meeting the 
special educational needs of educationally 

deprivctl children. (9) 

This is clearly a case where the objective is to affect 
the process domain. The special incentive grants are not, however, mvarded 



* v.x)rds in brackets paraphrase provisions of the act 
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on the basis of improvmont in education o£ the disadvantap.ecl. The procedure 
that is foUowctl instead includes the determination of individual state 
indices and a national index of effort. The indices of effort are ratios 
(expressed in per ccnta) of expenditures of all non-Fc^\eral money spent on 
public elementary and secondary education to total personal income. Those 
states whose indices exceed the national index are entitled to a special 
incentive grant. 

Although the special incentive grants are given to 
support compensatory e^lucation, the basis of reward is total ectucational 
expenditures. This has been criticized, for example, by Wilcnsky (46) as 
an inappropriate use of incentives to promote compensatory education and 
also on the basis that individual districts can do little to contribute to 
the state index. 

^fore ijnportantly for the purpose of this paper is 
the fact that, while the goal is in the process domain, the results (funding) 
and measurement (computation of indices) are in the input domain. 
C. Reward 

Tliis section discusses the third of the three key aspects of 
incentive programs - namely, the reward. Three aspects of rci^rd rarrant 
discussion: (1) the nature of the reward, (2) the timing of the reward, 
and (3) the mechanism of delivery. 

1. Nature of Reward 

All of the examples of incentive programs given in this 
paper used monetary rewards, and in tlie case ^f school personr.el, primarily 
salary or bonuses. ^vTiile it may be thd case tliat for programs in which an 
organizational entity is the target only monetary reivards are workable, it 
has been noted (Jung, Wolfe, and Lipe (21) among others) tliat other material 
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rewards, as woll as social rcu-aids (e.g., praise), hive been used. In a study 
concerned with professional Rro\rth of teachers, Cory (8) found that teachers 
considered incentives other than sal/iry useful. These additional incentives 
included among other tilings: (1) payment of expenses and paid leave to attend 
conferences, (2) fair and objective evaluation, (3) teacher participation in 
f emulating educational policies, and (4) valid opportunities to work on 
curriculum revision. 

This notion of re\%'ards other than those of salary or of 
a monetary nature touclies on some wrk reported by Fuller and Miskal (13) , 
who examined a recent industrial theor>' of wrk incentives in the context of 
an educational organization. This wrk is discussed more fully in Section IV D 
below. The point here is that rewards in incentive programs may need to be or 
should be other than monetary and may require that incentive programs be 
described on the basis of the nature of the reward, as well as on (all or 
some of) the dimensions suggested above. 

2. The Timing of the Reward 

A major variable in incentive learning studies is the 
temporal relationship between the occurrence of desired behavior and receipt 
of reward. Jung, Lipe, and Wolfe (21, p. 56-59) identified two nsaj or models 
in use with students. One of these was termed the "microincentive" model , 
the other the "macroincentive" model. The fomer was characterized by 
incentives dependent on small increments of student performance; for 
example, correct responses within a lesson. Under these conditions the 
time lag from occurrence of desired behavior to receipt of r«%'ard is qiiite 
short. The latter (macro incentives) is characterized by incentives on vcr>' 
large units of student performance; for example, test gains over a school 
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In nearly all incentive programs the rward Is dolivored 
by a suporordin?to position in tho system (e.g., the superintendent) to 
subordinates in the system (cug., teachers). Colcjnan makes the point that there 
seem to be two rays In which incentives may be provided. One Is that just 
noted; tho other is through a change in the strucUire of competition, such 
that rewards are not basal on a superior's evaluation, but on success in a 
competitive structure.,.," (7, p. 23) Coleman also suggests that incentives 
resulting from a change in competitive structure are better accepted than 
incentives dependent on a superior's evaluation. 

In this view, the notion of alternative schools, or 
alternative means of parental selection of schools - for example, a voucher 
system - is a fu idamentally different way to introduce incentives. 

In this context on^j of tho criticisms of the OHO perfor- 
mance contracting study may bo viewed in a different light. Saretsky (33) 
noted that there \-i&s a "'John Henry" effect; tliat is, the teachers of the 
control groups knew tliat there v'^s a race on, and the control groups performed 
atypically (better) . This suggests that perhaps tho competitive structure 
was altered by the introduction of a potential alternative - namely, performance 
contracting. 

The changes proposed by Johns (20) are dependent on a 
superior's evaluation; that is, the state evaluating districts. The evaluation, 
hoivever , is a competitive one; that is , an average level of performance 
determined by all districts is a baseline which individual districts must 

exceed to obtain the rc^-ard. 

This question of the mechanism through which the retard 
is delivered or obtained may be further complicated in those cases where, a 
strong teachers' union is involved. It is entirely possible that teachers 
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nay perceive or the imion may wish to take a posture such that the teachers 
would perceiyp , the union as has ic to the mechanisms by wli ich their incent ives 
are obtained. In any event, it seems reasonable to believe that this issue of 
mechanism is an important aspect of incentive programs. 
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III. A CONCEPTUAL PRAMRVORK 

The puriwsc of this section is to propose a concept for incentive 
programs as well as a conceptual framework to further Investigate incentive 
programs. In adcUtion, some of the iinplications of the framewrk will be 
examined. 

A. The Basic Concept 

The previous section examined a number of incentive programs in 
education. In addition, the word incentive is used by a nurriber of social 
science disciplines, * It also has a common meaning or usage/ generally a 
reward to be earned as a result of a special effort ; or > something which 
motivates or incites one to action* Within the social sciences the word 
incentive, while in a few limited cases having sme special technical 
connotations, is consistent with the common usage. Tlirce of the social 
science usages warrant further discussion. 

The first .special usage is that by oconmists in the context of 
"tax incentives," In this u'ie the general intent of the incentive is to 
prcMote or stimulate production or development, usually by providing a tax 
benefit to business organizations for meeting some special conditions: a 
reduction or elimination of taxes, for example, for some time period to a 
new or relocated company in an undeveloped area. Incentives in this context 
are aimed at organizational entities rather than at individuals. 

The second special use is that in psycholog>% particularly in 
learning theory whore the term is used to**, . •describe the learning process 
dependent upon the stiDiulus-response-goal sequence..." (25, p. 3), where goal 
is the reward/punishment. This meaning is, of course, the learning paradigjn 



* Appendix A of this report contains definitions of incentive taken from a 
O variety of sources. 
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used by the behaviorists and generally serves as the model for studies of 
incentives in the classrooms, liven in this special izc<l uso of incentive, 
hwevor , the word is swnctimes used to refer to tho goal (reward/punishment) 
itself. As implied by this definition, a negative reward, i.e. , punishment, 
may be use<l. In such cases the expectation is that attempts will be made to 
obtain results that will reduce or eliminate the penalty. There is in this - 
definition a coupling between the stimulus/ response and the reward. 

The third special use of incentive that needs to be considered 
is that used in contracting. In this context the term "incentive contract" 
means a contract that specifies a set of individually identifiable, acceptable 
outcomes and a range of payments, one for cacli outcome. T\n"o aspects of this 
usage are irajiortant: first, is specification of all acceptable outcomes; 
that is, all of the outcomes must satisfy in some respect the general goal 
of the contract; and second, some outcomes are better than others and the 
better the outcome, the greater the reward, 

In sumnary, all of the usages of incentive involve a reward to be 
given for some action on achieving smo result. In some cases the reward may 
be offered to organizational entities rather than individuals. There is a 
coupling of the reward with the behavior required or tlie results to be obtained, 
Finally, the reward may have a range of values which may be associated with 
a range of results. Thus, defining incentive from the usage in the social 
science disciplines leads to the same considerations as used to examine the 
several exanples of incentive programs in education in the previous section. 
Considering both the definition of incentive and its use in education, a 
basic concept of incentive programs may be formulated as follows: 
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Incentive programs havo a goal and 
offer a reward or set of rewards to 
individuals or organizations for 
undertaking certain actions or 
achieving certain results. These 
actions or results presumably further 
the goal, and thei^e is an implicit 
or explicit association between the 
reward (s) and the result (s). 

This concept may be separated into three parts for the purpose 

of constructing a conceptual framework. These three parts are analogous to 

the three characteristics used in the previous section to discuss the example 

programs; they are: 

(1) The individuals or organizations who may receive the 
reward 

(2) Tlie goals/results 

(3) The reward. 
B. A Conceptual Framowrk 

The conceptual framewrk proposed herein will hopefully serve 
three purposes. First, it may serve as a checklist in describing or examining 
an incentive program. Second, it provides a simple .taxonomy for classifying 
such programs. Finally and most importantly, it may serve as the framework 
for a systematic, long-range investigation of incentives and incentive 
programs and their place in the organization and governance of public education 
in the United States. 

1. Targets of Incentive Programs 

The targets of incentive programs are defined as the enti 
ties that will, under the proper conditions, receive the reward. The targets 
of an incentive program need to be specifically identified and provide one 
variable for classifying a program. The first distinction to bo made is 
whether the target is individuals or an organizational entity. Within the 
pnj/->-atcgory of programs with individuals as targets, the program may be further 



27. 

defined by the class or combiaition of classes of Individuals who are the 
targets. Programs in the other major class, tlmt is with organizational 
targets, may be further categorized by two considerations: (1) the level of 
tho organizational target and (2) the nature of the change to be sought. 
The details of this variable of the conceptual framework are outlined below* 

A. Individual Targets 

1. Superintendent (School System leader) 

2. System Administrative Personnel 

3. Principal (School Leader) 

4. School Administrative Personnel 

5. Teachers 

6. Parents 

7. Students 

B. Organizational Targets 
!• Organizational Level 

a. Individual School 

b. School District (LFA) 

c. State 

■ 

2. Nature of Change 

a. Organizational (form or size) 

b. Structural or Operational Change 

It is conceivable that a program might have both 
individual and organizational targets. In this case, of course, both major 
categories would have to be used either to describe or classify such a program. 

2. Goals/Uesults 

The incentive program concept defined above states that 
the results to be obtained ''presumably further the goal'' of tho program. The 
implication of this is that the goal and the results are consistent. In this 
context it may be well to point out that the wrd results is intended to convey 
the idea ''those conditions vMch are to be net and for which the rerard vdll 
given." ITiis latter idea carries with it an imwrtant imi^lication - namely, 
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that there is a method of determining when conditions have been mot; or, in 
other v-ords, a method of measurement, and equally important that tho method 
is agreed upon by all parties. Thomas and McKinney (43) note that one of 
tho criteria for merit pay plans is consensus by the participants on tlie 
state-of-the-art and tools and techniques required for performance measure- 
ment. Tcmpleton (36) states that one of the major criticisms of merit pay 
plans is the lack of objective standards for teacher ratings. 

All incentive programs are based on the assumption 
that at least ono procedure (method or technique) exists, which, if used, 
will obtain the desired results. This is so, even in those cases where 
the procetlure is either not spelled out or is left to the choice of the 
targets of the program. For example. Carpenter and llall (4) conclude that 
in the cases of performance contracting studies for their report, in all 
cases the contractors were to be paid for results and chose individualized 
instructional teclmiques as the preferred metliodology. The OEO study of 
performance contracting left the choice of method to the contractors or, 
in the case of the incentive only portion of the study, to the teachers. 
Miller (28) has stated that this exclusion of method as an explicit variable 
was a serious omission in the research design. 

The foregoing discussion leads to the conclusion that 
there are four aspects or characteristics of incentive programs which need 
to be considered. These characteristics are: (1) goals, (2) procedure, 
(3) results, and (4) measurement. Pinally, in considering these cliaractcr- 
istics of incentive programs as applied to education, the concept of 
education as a system with domains of inputs, processes, and outputs will 
be used, 

o 
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This goals/results variable of the conceptual framcTOrk 
dealing with the characteristics of incentive prograjns and their relationship 
to the educational system may best bo defined by a matrix, with one dimension 
the program characteristics and the other the domain of the system, as shorn 
below: 

Domain of Educational System 

Incentive 



Program 
Characteristic 


Input 

(i) 


Process 

(p) 


Output 


Goal (G) 








Procedure (P) 








Results (R) 


"i 


■v 




Measurement 0^0 


"i 







The foregoing matrix permits clarification or extension 
of some of the concepts discussed above. First, it has been suggested tlvat the 
goals and results ought to be consistent. It follows that the measurement must 
also be consistent. Tliis may now be restated - *'the goals, results, and measure- 
ment method must all be in the same d(»nain.*' Second, a cc»nplote program requires 
that all four of the characteristics must be specified. It may be in sme cases 
that the choice of method is to be left to the targets. Such a degree of freedom 
should be duly noted as an aspect of the part iailar program. 

Finally, it should be noted that in the cell of the matrix 
formed by the intersection of procedure and output there is no entry. It will 
be recalled that procedure is tlie method used to bring about the desired results; 
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consequently, such a cell may not contain an entry as no method for output 
can exist. Further, if the input/procoss/output dimension is considered to 
b© a sequence, then the procedure must not bo later in the sequence than the 
goals/rcsults/measurcment entries. This implies that if one wishes to alter 
output, then either the inputs to the syst«n or the processes used by the 
system must be altered. If the goal is to change an input, then the pro- 
cedures must be from the set that is used to determine system inputs. 

The Title I program described in Appendix B and cited 

earlier in this paper had as a goal the improvement of educational programs 
for the educationally deprived at the level of the school district (LFA), 
The incentive grants, however, were given on the basis of funding considera- 
tions to the states for subsequent distribution to the districts, Tliis 
program would be thus classified as an organizational target (the LEA) with 
goals in the process domain but with results (conditions for reward) in the 
input domain and with reward distribution depentlent on the mechanism of a 
superordinate and partially dependent on the actions of an organizational 
level (state applications) different from the target. As defined by this 
(the goals/results) variable of the conceptual framework, the Title I program 
vwuld be considered to be an inconsistent program on the basis of having 
goals in one domain and results in another. There is also an clement of 
inconsistency in having two levels of organizational targets, the state 
and the I£A's. 

The USOK "Incentives in Education" project, which 
did not deal explicitly with tlie instructional methodology (i^roccdure) to 
be used to improve student learning (output goal), would be defined as an 
incomplete program. 
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3. Reward 

The final variable of the conceptual framcKork is to 
use the characteristic of the re\^rd to describe or classify incer.tive 
programs. Tl^ree characteristics of reward are used to distinguish among 
incentive programs, Tl\ese three characteristics are: (1) the nature of 
the reward, (2) the mechanism of delivery, and (3) the tijning. Each of 
these primary characteristics is further subdivided. 

The nature of the reward is distinguished on two 

dimensions. Pirst, i^hether it is a monetary or some other form of reward, 
and second, whether the rcmvd is given individually or on a group basis. 

Tlie mechanism of delivery uses only two subdivisions: 
whether the reward is determined by a superordinate in the system or whether 
there is a cliange in the competitive structure. 

Finally, the timing characteristic has two subcategories 

These categories follow the notion of the microincentive or macro incentive 

models defined by Jung, Lipe, and Wolfe. (21) 

It is envisioned that this characteristic be used when 

a student incentive model is combined with one of the other individual target 
models. Qirrent practice and research on teacher incentive programs use 
exclusively a macroincentive model . ^vhile it may be argued that this is 
the only practical approach when financial incentives are used, development 
of alternative re^rard models (see IV D below) may show that microincentive 
models with teachers o* other personnel are possible. 



32. 



Tho details of this variable are outlined below: 



A, Nature or Keivarci 


Basis 




Typo 


Individually 


Group 


Money 






Other 






B. Mechanism 






1. Superordinate 

2. Change in competitive structure 




C. Timing 






1. Mlcroincentivo 

2. Macro incentive 





o 
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IV, CONCLUSIONS m\) REO^MnNnATIONS 
A. Neod for I-rajnework 

In all cases incentives generally are taken to mean the re^^rd 
to be given/received as a consequence of a prespecif ied behavior or achieving 
a prespecif ied standard, In spite of this conmon (and commion sense) meanings 
the rtK)st striking characteristic of incentive use is the vast variety of and 
differences among programs which fall under the incentive rubric. This 
diversity (yet coirmonness) results in some confusion in discussing or consider- 
ing incentive programs and research on incentive programs. 

If one accepts the general definition of incentive, the wide 
variety is not only justifiable but inevitable for two reasons. First, 
there are a great number of places within the educational systm where 
incentives may be applied. Second, it is not likely that economists, 
sociologists, and psychologists, for example, who all use incentive in their 
respective disciplines,, will jointly agree on nc\7 and/or different words. 
As the various disciplines attack problems in education, it is more probable 
that the terms and their meanings of the different disciplines will be used 
in an educational context. 

There is a conspicuous lack of an overall conceptual approach 
which provides a framework for a systematic investigation of incentives in 
education. Although it does not seem to have been implemented, Jung, Lipe, 
and Kolfe (21) did suggest a general framowrk for incentive programs 
targeted at individuals. Tlie programs they were concerned with were 
generally limited to those in which student learning (output domain) ^\^s 
the criterion variable. As suggested in this paper, such incentive programs 
comprise only one categor>' among many. 



ERLC 



It seems clear tliat a fra -.ewrk is needed. It is strongly urged 
that a conceptual framework, whether it be the one suggested in this paper, 
a variation thereof , or a new and completely different one should be established. 
The framework should meet, in as far as is possible, the following general 
criteria: 

(1) Permit the categorizing of incentive programs \dth a minimum 
of overlap and ambiguity. 

(2) Provide for identification of those categories of incentive 
programs and facets thereof of most interest for further 
©cplorat ion and/or research. 

(3) Provide guidance to systematic exploration of programs of 
utmost interest. 

B. Objectives mi Measurement Problem 

Many, if not all, of the advocates of incentives mean the word 
to be applied to those cases where outc<xnes of the educational system 
(e.g., student learning) are the desiderata, and for which rewards will be 
made. This is generally consistent with the position that incentives 
may be considered to be an aspect of accountability. More generally, however, 
incentives need not be applied only to outcomes of the educational system, 
although the behavior or state for which the re\^rard will be given must be 
objectively assessable. This requirement brings the measurement problem 
and the related issue of goals or objectives specification into sharp focus. 
In part iailar, those incentive programs for which educational systems outcomes 
are the criteria are crucially dependent on resolving two issues. First, the 
specification of educational objectives and second, adequate or acceptable 
measurement instruments. 
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Jung, Lipe, ancl Ublfc (21, p. 3) raise three questions: 

(1) Can educational goals be identified which aro 
significant yet provide suitable criteria for 
incentives? 

(2) Will major parties in the educational process allow 
. . . external incentives contingent up^^ 

of such goals? 

(3) Can incentives be more effective than existing 
prograips in goal attainment? 

Thomas and ^!cKinney (43), who articulate the view that merit pay 
(and incentives generally) may be viewetl as part of accountability, identify 
several criteria relative to accountability plans. (43, p. 39 ^ 40) Tliese 
criteria include: 

(1) Agreement on the objectives of the plan 

(2) State-of-the-art, including participants consensus 
relative thereto, on the tools and techniques of 
the required performance measurement 

(3) Willingness of the stakeholders (i.e., major parties 
in the educational process) to participate. 

Stucker and Hall (39) state that program/product specification 
and objective measurement of performance are essential to performance 
(i.e., incentive) contracting. They review several statements of etlucational 
goals and objectives and suggest their general inadequacy for a perfoimance 
contract. 
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Tanploton states that one of the major criticisms of merit pay 
plans is that there are no objective standards available for teacher rating. 
(42, p. 2) 

Stake (31) reviewed and analyzed the problans of testing in 
performance contracting. His analysis of the hazards and pitfalls in that 
context is applicable to any incentive program which depends on assessment 
of educational system outputs. Cohn and Millman (6) note that before one 
can provide incentives to schools for more or better output, a specification 
of that output and its measurement are necessary and propose that a start 
may be made. 

In essence, Cohn and Millman (6), while admitting the state-of* 
the-art is far from satisfactory, propose an overall index measure of education 
output, combining several separate indicators of output. The formulation tliey 
present is the notion of a generalized educational product ioa function in 
which a set of educational objectives (i.e. , outputs) is a function of the 
interaction of a set of educational Inputs (e.g., budget) with a set of non- 
sc1k)o1 factors (e.g., tamily Socio Econcmic Status). They give a specific 
example of how such an approach might be employed using data from Kuhns. (22) 
^Vhile this formulation provides an approach to the problem of a generalized 
output measure, it is limited to secondary schools. Itore importantly, it 
does not resolve the question of agreement to objectives or adequate measure- 
ment instruments. 

In summary, the specification of objectives and the demonstrated 
means of measuring tliem, both crucial to incentive programs dealing with 
educational system outputs, are far from satisfactory. 
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C. Lack of Theory 

Work on incentives to elate has had a minijnal theoretical foundation 
uhdetgirdinc hoth practice and research. Jung, Lipe, and Wolfe (21) as noted 
earlier distinguish between two student incentive models - the microincentive 
and macroincentive - and point out that the latter has almost no research 
history. It may be argued that the microincentive model is the beliavioral 
psychologist's learning paradigju as applied to the school situation, and, as 
such, has a theoretical basis, whereas the macroincentive model does not. 
It is the latter model, however, which has been used in the most publicized 
studies of incentives in learning, including teacher incentive programs. 

A macroincentive model, it will be recalletl, usually involves a 
considerable delay between the results (occurrence of desired behavior) and 
receipt of reward. A merit pay plan, for example, might use observation of 
teacher performance as a measure in 1 year and a salary increment, in monthly 
installments (as a reward) in the following year. Theoretical formulations 
regarding incentives in learning psychology do not typically cover such 
extended delays between behavior and reward. Tlieorizing about such a model 
would, it seems, suggest invoking such notions as "internalized value systems" 
or "delayed gratification concepts." No such theoretical formulation as 
applied to incentive programs lias yet been done. 

The lack of a theoretical basis may be noted elsewhere. Stucker 
reviewed the major theoretical articles bearing on performance contracting 
and states "no general definitive statement of that theory has been found, 
but most of the major elements of the theory arc presented." (8) Stucker 
then sunmarized the tw articles he deemed most pertinent: one by Simon (30) 
and one by Yowell. (47) According to Stucker, Simon's work is concerned 
with sales contracts (contracts for results) and employment contracts (contracts 
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for labor resources) aiid the trade-offs between them. Stucker suggests Simon's 
discussion may bo rephrased to deal with fixed or performance contracts. 
Stucker' s review of Yowell's work makes thft following points; 

(1) It is a general discussion - theoretic model. 

(2) It refers to only two parties, a principal and an 
agent . 

(3) It assumes the relationship is established for the 
benefit of both, 

(4) It focuses on the ways a principal (employer) attcnqDts 
to guide the actions of an agent (employer or 
contractor) . ' 

(5) It examines the extent to which the principal 
can guide the agent by rewards conditional on tho 
agent's results. 

(■6) It assumes the agent will try to maximize his results. 

In this context of theory two other approaches may be mentioned. 
These approaches deal with the relationship of individuals to organizations. 
The first is ^^arch and Simon's concept of "inducements and contributions." 
(26) Tlio second is Ftzioni's concept of "compliance." (11) These t^vo formu- 
lations suggest alternative ways of conceptualizing the mechanisms of reward/ 
punishment in an organizational context. 

In sunmary, there are several theoretical approaches which bear 
on the use of incentives in education. It would appear tliat a major issue is 
to rationalize these alternative theoretical formulations. 

An effort to examine the variouf theories (some of which have 
been identified in this paper) tliat apply to the use of incentives should be 
undertaken. Some of the steps in such an effort should be; 

id 
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(1) A mapping (tliat is, where and how they apply) of the 
various theories onto the conceptual framework should 
be atten^ted 

(2) Such a mapping if at least partially successful should 
yield information on: 

(a) Improvement of the conceptual framework 

(b) Research areas where minijnal or no theory is 
available 

(c) Overlaps and gaps among the theories 

(3) An attempt to extend, exi)and, cr clarify theories 
to account for overlaps and gaps. 

D. Boundary Problems 

Some of the issues and problems of both practice and research on 
incen. vei? are also problems in other areas of education. A good cxara{>lc of 
such a problcan is evali^ation and measurement which arises in a variety of ways 
in almost all areas of education. Another exair^ile is that of student incentive 
models, of special interest in learning studies in educational research, which 
may be combinud with various of the individual target incentive programs. 

A third example, but of a different kind, is that of ••differentiated 
staff ing.'^ As noted earlier, Tliomas and McKinney i^Z) consider differentiated 
staffing to be a variation of merit pay plans. Differentiated staffing plans, 
hoxvever, not only involve differential pay but also differential teacher roles. 
One of tlie proposed patterns of differential staffing is a hierarchical structure. 
For example, the "Clark" plan proposed for Washington, D.C. and the Temple 
City, California plan are of this type. Although it is true that American 
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public olcmontary and secondary schools arc in general hierarchies, this 
concept of differentiated staffing extends the hierarchy to the point of 
tcachor/student interaction. Although some organizational theorists have 
considered hierarchies in bureaucracy, little or no conceptual wrk regarding 
the place or impact of a teacher hierarcliy in the educational milieu has 
been done. Although different iate<l ftaffing may be studied as an incentive 
program,' it seems equally, if not i^wre imjiortant, for differentiated staffing 
to be studie<V from an organization factors point of view. 

H. Alternative Rewards 

Although student incentive studies have used a variety of rewards, 

those incentive programs for school personnel, both operational an<V experi- 
mental, have used exclusively monetary rewards. One study. Hooker and 
Sumnerfield (18), compared professional gro\rth patterns of teachers from 
high incentive with teachers from low incentive districts. In this study 
the incentive aspects were salary schedules based on experience and academic 
preparation. Their results seemed to indicate that the amount of incentive 
had little if any effect on the nunber of advanced credits taken by teachers. 
Their findings do indicate that perliaps other more pervasive and subtle 
incentives, sex role expectations (e.g., males as school administrators) , 
for example, were involved. 

As noted earlier, Cory (8) identified rewards other than monetary 
which both teachers and principals thought valuable in providing incentives to 
' professional gro^rth. A paper by Fuller and rUskel (13) bears directly on this 
problem. Tlicy examined in an educational context a recently- developed 
industrial theory. This theory uses the concept of sources of work attachment; 
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that is, features of the work environment to which human behavior is directly 
related. 

The idea is to find which features of work attacliment are sources 
of satisfaction/dissatisfaction and relate incentive thereto. Fuller and 
Miskel used a questionnaire of 124 items (falling into 8 categories) of sources 
of wrk attaclfnent. Each item Kid three responses: satisfied, indifferent, 
and dissatisfied. T\\e questionnaire was administered to 508 staff members in 
one school district. It was found that although all teachers responded to 
certain work features in similar ways, tliree distinct groupings of teachers 
(satisfied, indifferent, and dissatisfied) also responded differently to other 
work features. Fuller and ftiskel suggest that an incentive plan which provides 
for one set of incentives in conmon and three different sets of Incentives 
for the three groups of teachers may be indicated. 

In view of some of the studies which seem to indicate the question- 
abaility of monetary incentives to teachers, a more thorouph exploration of 
alternative rewards seems indicated • 
F. State- of- the- Art 

Incentive programs in education do not meet reasonable standards 
of exiierimental studies, in the sense that Campbell and Stanley (3) define 
the criteria of experimental or quasi -experimental studies. The most widely 
publicized use of incentives - the OFjO performance contracting study - has 
been sliarply criticized by Miller (2f;) as having been a failure in experimenta- 
tion. Tlie USOD study "Incentives in Education" (see 1 and 31) also lacked the 
rigor required of an experijnent. Both of these studies, for example, lacked 
randomization for key features of the design and in addition, focused 
primarily on under- achieving, disadvantaged students. Thus, although both 
studies report conclusions on the value of incentives, it may he the case 
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that these conclusions are not oven general izablo to other disadvantaged 
students, lot alone to the general public scliool population ♦ 

There seem to have been no exporijnental studies undertaken or 
designed to examine or permit generalization to individuals other than 
students. In no sense was either the OK) or tl\o USOE study designed In 
such a way as to be generalized to teachers • Certainly there are a minimum 
of programs applying incentives to principals and administrators in the 
system. 

Those programs focusing on organizational entities not only 
have not been examined in an experimental context, but there is also a lack 
of case studies. 

Although case studies lack the rigor of experiments, they may 
serve to provide insight to important variables of the problem. In addition, 
case studies may serve as preexporimental vehicles for examining theoretical 
or conceptual formulations of the problem. 

Considering the state of experimentation on incentives, the 
situation with regard to theoretical foundation for incentive programs, and 
the general lack of a well -documented and organized body of knowledge about 
incentives in education as broadly conceived as in this paper, then a group 
of well-done case studies may be the best starting point. 

Assuming tlvat a conceptual frameu'ork is adopted as suggested 
above, it would be useful to undertake a series of case studies including 
the use of field methods such as participant-observation and micro-enthography, 
for example, aimed at documenting as many of the categories of the framework 
as possible. 
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APPENDIX A 
SatR DEFINITIONS OP INCEITTIVE 

I. (Xtim USAGE 

■ ■ • * . 

In coitmon usage incentive is generally a special re^vard which 
encourages people to special or extra effort. Tlie dictionar>' (38) defines 
it as: 

1. (adj) Inciting; encouraging or nvoving; rousing 

to action; stimulative. 

2. (n) Tliat which incites or has a tendency to 

incite, to determination or action; 
motive, money and pride are incentives 
to action. 

Synonyms include: goal, stimulus, and inducement 

II. ECONOMICS 

As used in economics, the meaning is consistent with the comon 
usage except that the reward is generally monetary in nature. For example, 
Hanson (17) gives the following definitions: 

Incentives 1. With reference to wages, a system 
of wage payments which offers an inducement in^tlie 
form of a bonus to encourage worlcers to maintain a 
high level of output. Opportunity of promotion is 
another type of incentive ... 

2. With reference to the taxation of 
wages and profits, it moans a tax system designed 
to encourage an expansion of output. 

These definitions are consistent with others offered in dictionaries 
of economics; for example, Greenwald (16) and Sloan and Zurchcr. (35) In 
the same general vein as tax incentives is the use of incentives to aid 
underdeveloped areas. These incentives often, although not alts-ays, include 
a taxation structure which favors a new (or expanded) company during its 
initial years. Incentives in tliis context are generally aimed at organiza- 
tional entities, ratlier than individuals. 
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III. PSYQIOLOGY 

In mcxlcm psychology incentive is generally limited to an aspect of 

learning having to do with a re^vard/punishment following a response. For 

example, Logan and Kagner (25, p. 3) define the concept of incentive as: 

... The tem incentive is here used to describe 
tlie learning process dependent upon the stimulus- 
response-goal sequence. ... Incentive is most 
analogous to what in ever)^day language might be 
called an expectation that re^N'ard and punishment 
will follow a response ... The term incentive 
is also widely used to refer to the reu^rd itself 
especially when the reward is visible to the 
subject. Incentive is here reserved for the 
learning process dependent on reward of punish- 
ment because the inportant thing is for the 
reward to become internalized; 

Psychological Abstracts lists incentive in its index with the note 
"see also >fotivation, Rerard, Reinforcement." 

The word incentive is, of course, much used by the behavior ists 
since they use the stimilus-response-goal learning paradigm. 

Industrial psyc} : legists use incentive generally in the context of 
monetary reward in the sense of the economists* incentive rage systems. 

IV. SOCIOIjOGY 

Sociologists generally follow the definition used by psychologists. 
Hoult (19) defines incentive under "motive (motivation)." 

In the general sense any condition of an organism 
promoting selective activity ... in modern behavioral 
psychology all determiners of behavior . . - in the 
social sciences often used synonomously with 
incentive .... 

V. rnVTRACTING 

Stucker, in witing on performance contracting in education, states 
(38): "Contracts that allow for a range of acceptable outcomes and specify a 
range of payments corresponding to these outcomes are temed performance or 
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Incentive contracts." Stucker also cites a paper by Yowell (47) which defines 
Incentive as: 

Incentive: A conditional roiv-ard provided only if 
some stated result is achieved. 'You will be 
retarded with A only if you accomplish B. ' 

VI, EDUCATION 

In education, incentive is used in ways consistent with the foregoing 

definitions. For example, Good (15) gives the following definition generally 

consistent with economists' usage. 

pay, incentive rate of (school administration) the 
percentage of rate above base pay that should be 
earned by the average employee who has been properly 

selected and trained for his work, when he meets a 
fair standard of perfomance. 

Also in fiood (IS) : 

incentives, the factors and forces that incite or 
motivate one to action. 

The latter, of course, being more the comon usage and generally 
consistent witl\ the use in psychology. 

The mucation Index lists under incentives primarily incentive pay 
systems and provides a cross-reference to "reward and punishment." Tlie ERIC 
subject index lists incentive grants and incentive systems. The former 
generally has to do with distribution of state aid; the latter with classrocm 
learning, and some of these using a behavioral approach to classroom learning. 
VII. SLP'MARY OF DEFINITIONS 

Nearly all of the definitions given above are but application of tho 
common meaning in a special context; e.g., economics. Tl\e exception seems to 
be in the use by beliavioral psychologists to mean the concept of stimulus- 
responsc-goal (rcward/punishnent) , but even here incentive is often used to 
mean zhe reward (punislment) . 
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' APPHNDIX B 

EXAMPLES OF INCENTIVE PROGIWtS IN EntJCATION 

This appendix describes several incentive programs as examples of 
applications in etlucation. Tl\e appendix is organized on the basis of the 
conceptual framework proposed and discussed in Section III. The first 
division is made on the basis of whether the program has an individual or 
organizational target. Programs having organizational targets are further 
subdivided on the basis of the nature of the change sought - either a change 

to organization or a change to operation of structure. 

Programs having individual targets arc organized by the portion of 

the educational system domain (input, process, output) which contains the 

program goal. 

Each program is described in terns of the goal, the procedure, the 
measurement, and the re\^'ard, and finally described briefly in accordance 
with the conceptual framework. 

I. ORGANIZATIONAL TARGETS 

A. Crgani national Change . 

During the last lialf century one of the prime movements in 
American public education lias been the movement toward more and larger com- 
prehensively-organized school districts; that is, districts which offer K-12 
or 1-12 grades under a central administration. The first example presented 
in this section is of the current California statewide program to promote 
the formation of comprehensively organized school districts. 

Colrn and Millman (6, p. 88) state: "considerable evidence 
demonstrating the existence of substantial scale economics in public 
(especially secondary) schools has been presented in recent years..." and 

o 
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go on to suggest that changes In size of schools (school districts) ought to 
affect economies of scale, i.e., per nupil cost relative to size. The second 
example is of a program proposed by Cohn and ^tillJnan to modify school organiza- 
tion to affect economies of scale. 

1, California Incentive Program for Unified Scliool Districts 
In the State of California, comprehensive districts are 
called unified school districts. The California Pducation Code of 1959, as * 
amended through 1972, encourages the formation of unified school districts 
throughout the state and specifies: (1) certain procedures to be folloKcd, 
(2) a means of assessing conformance, and (3) a monetary reward. 

a. Goal. The California lilucation Code (45) in 
Division 5, Chapter 10, Article 1, Section 3100 
states (in part) : 

Legislative intent - 

It is the intent and purpose this 
division be utilized primarily for the 
formation of unified school districts 
and that this form of organization be 
ultimately adopted throughout the state. 

Section 3131.5 provides furtlier evidence of 

the goal as it includes the folloidng: 

In order to establish a system of unified 
districts tliroughout the state, (appro- 
priately approved plans) shall supersede 
any organization or reorganization accom- 
plished under other provisions of this 
code. 

Procedure . Division 5, Chapters 1-10, 
Articles 1601-3587 of the California Fducation 
Code (45) deals v.'ith organization and 



Q* Words in brackets paraphrase provisions of the section. 
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reorganization of school districts and establishes 
the procedures to be followed. Tlie procedures 
for foiming unified school districts include: 

(1) The establishnont of county committees 

(2) The development of coimty-wide or in 
some cases multi-county plans 

(3) Voter approval of such plans 

(4) Voter approval vdthin the affected 
districts of each nav proposed unified 
district 

(5) Submission of plans to and approval or 
disapproval thereof by the State Board 
of Fxiucation 

(6) Provisions for plans to be made at the 
state level if the local areas do not do so 

c. Measurement . The various aspects of the 
unification process are administered by the State 
Superintendent of Riblic Instruction (Chapter 9, 
Section 3001) in accordance with rules and regu- 
lations of the State Hoard of Fxiucation (Chapter 9» 
Section 3002). Criteria for the formation of 
unified districts arc established by Section 3100 
and include among other provisions: 

(1) Adequate enrollment 

(2) Adequate financially 

(3) Substantial coiminity identity 

(4) Docs not promote racial or ctlmic 
discrimination. 
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^* Rg^ard> Division 14, Chanter 2, Article 2.1, 
Sections 17671 and 17677 provides for additional 
state aid for those districts in conformance with 
the provisions of Division 5, Chapters 9 and 10 as 
determined by the State Board and certified by the 
Superintendent of Public Instruction. Under these 
articles each district must have an average daily 
attendance (ADA) of not less than 2,000. Tfie 
additional aid is $20*00/ADA. The 1969 amendments 
to the code provide for a foundation program of 
$755/ADA for elementary school pupils and $950//J)A 
for secondary pupils. The incentive aid is about 
2.6 percent for elementary pupils and 2.1 percent 
for secondary pupils. Since plans for unified 
districts may be made at the state level if the 
local areas fail to do so, there is also a punish- 
ment; i.e., loss of local control, 
e. Conceptual Frameav'o rk Discussion . TJds program 
is targeted at an organizational entity • the school 
district - and seeking a change in organization form. 
The goals, procedures, results, and measurements 
are all in the input domain. Finally, a monetary 
retvard, paid individually on the l>asis of a super- 
ordinate mechanism, is usetl. 
2. Propose<l Fx:onomics of Scale 
As noted earlier, Cohn and Millman (6) suggest that 
proper size of school districts may result in economies of scale. In dis- 
} fussing this issue they note that most of the studies IrKlicate a 
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U-shaped relationship between per pupil cost and school size, measured by 
enrollment. It follows that most schools are cither too largo or too small .... 
(p, 89) Their report goes on to note, however, that there is some Indication 
that U-shaped relationships may not be accurate, and that it is possible that 
costs might decrease indefinitely as scliool size increase. Cp. 91) Never- 
theless, they assume the U-shaped relationship in proposing a program of 

promoting economies of scale. 

a. Goal . The objective would be to promote school 
organization to obtain optimal-sized scliool districts. 
Cohn and Millman state "an explicit incentive 
structure in the state aid process would not only 
provide a certain degree of stimulus to change 
school organization but would also serve to focus 

^ attention on the scale issue." (6, p. 88/89) 

Three options are proposed: (1) A penalty 
factor, (2) An incentive payment for schools which 
take action to improve their cost posture, and 
(3) A combination of penalty and incentive. Tor 
purposes of example, only the first option (penalty 
factor) will be considered. 

b. Procedure . The procedure involves the assumption 
that there is a determinable optimal -sized school 
district. A study by Cohn is cited in which he 

found that a school of 1,653 (pupils in ADA) was 
the optimal size relative to per pupil cost. In 
essence, schools both larger and smaller than 
optimum would have greater per pupil costs. 
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The proceUuro involves determining a penalty 
factor " PF - which is based upon: 

(1) The unit cost {C^ for the optimal sized 
school district 

(2) The adjusted unit cost (cp for each school 
district 

(3) A scale factor (p) between 0-1 determined 
by the state legislature 

such that: 

PF * P (CfC^ would be the penalty factor for 
the ith school district. Obviously, the larger the 
p, the larger the PF. The unit costs could consider 
a variety of factors such as: 

(1) Average number of hours college semester 
hours per teaching assignment 

(2) Class size 

(3) Building value per ADA 

(4) Number of credit units offered 

(5) Median high school teacher salary 

(6) Bonded indebtedness per ADA 

(7) Average number of different subject matter 
assignments per high school teacher. 

c. Measurement . The measurement would be straight* 
forward, based upon school district reports 
containing the data necessary to compute the 
various costs and establish tlie cost of the optimui^- 
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sized schcK>l. These data would also include school 
size • pupil enrollment in ADA. 

d, Rqyard . In this case the reward is to be obtained 
by moving toward optimum size and reducing the penalty 
to be incurred. In operation, the penalty factor (PF) 
is used to reduce state aid by multiplyjijig the PF by 
enrollment - pupils in ADA (P.) and subtracting the 
result from the state aid (A.) for the district t 

^adj^r (^^^•^^ adjusted state aid for the 

ith district. 

Thus, if state aid were given on an ADA basis, 
say $600/AnA and the PF for a specific district 
were say $30.00/AIA, the district wuld receive 
only $570/ADA or be penalized 5 percent of its aid 
monies* 

e. Conceptual Franewrk Discussion . Although this 
program is targeted at an organizational entity, the 
precise target is not clear* The target is limited 
to the secondary level, but whether it is individual 
schools or districts is not clear, Th^e program is 
seeking a change in organization, specifically in 
size. The goals, procedures, results, and measure- 
ments are all in the input domain. The reward is 
monetary, paid individually, through a change in 
competitive structure but involving a superordinate 
meclumism. 
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B/ Structural or Operational Change 

Tiito examples of incentive programs to modify operation of public 
education are given* Tlio first oxnmj)le was a pro^vDsal to modify the method of 
distributing state aid based upon considering the relationship between school 
districts effort required to supix)rt a minijiiun education program and its actual 
efforts. The second example deals with an effort to modify the emphasis whic)i 
schools place uix)n education of certain educationally disadvantaged students* 

1. Ratio of actual effort/minimum effort as a mechanism 
to distribute state aid 

Johns (20) proposed to modify the Florida State Aid 
Program, a foundation program, by providing a portion of state monies by 
comparing the ratio of local (district) actual effort to support education in 
relation to the local effort required for a minimum program with state average 
ratio of actual effort to minimtm effort, This proposal to affect distribution 
of state aid monies differs from the Cohn and Millman proposal. The latter 
were directly concerned with affecting organization, whereas here the concern 
is with structure; that is, the mechanism or operation of local (district) 
support of education. 

Joluis makes clear tliat an incentive program sucli as he 

proposes must be developed in the context of a particular state and its aid 

program. The example which follows is taken directly from Johns' incentive 

program development for the State of Florida. 

a. Go a l , The goal is based on the belief tliat 

education must now go beyond minimum requirements. 

Jolins states: 

Tlie original purpose of the foundation 
program was to guarantee a minimum 
eilucation program ... It is now bolicved 
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that the state (mist provide) financial 
incentives for school districts to provide 
more than a minimum program (20, p. 4) 

The goal is then stated succinctly: 

The purpose of this incentive grant is to 
give additional state financial help to 
those districts that are willing to help 
themselves financially. (20, p. 0) 

b. Procedure . The procedure involves having the 
legislature establish state aid in two categories: 
(1) a foundation grant to be in accorilanco with 
existing laws and (2) an incentive grant to bo 
given to those which exert an effort beyond that 
required for the minimum foundation program. The 
procedure also requires obtaining appropriate data 
from the districts and computing appropriate ratios. 

c. Measurement . The moasurment is basically a 
mechanical process depending on data supplied by 
the districts . The computations involve : 

(1) Determining for each district the miniiiium 

- tax effort required to supi>ort the Minimum 
Foundation Program (Tj^) 

(2) netcrmining local (district) tax effort (T^) 

(3) Computing the local ratio " 

T 

n 

(4) Computing the state average ratio It " i«l R 

(5) Computing district multiplyer - Ti^ ^ 

if iy-1.0 district is average r^ 
or above in effort. 
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d. Reward. The «ictual rward would depend on the 
amount established by the state. The computations 
are comparatively simple. 

(1) Compute the base ainount of the incentive 
grant (G^), in Florida the number of 
instructional units (1^) in the Minijnum 
Foundation Program mult ipled by the 
incentive aid (Aj) per I^, Cy^" I^Aj 

(2) Compute the actual incentive grant (Gj) by 
multiplying the % by the Gj - % \ 

Johns establishes limits on both Rgand I^; in 
the latter case, an upper bound (r^- 1.100) means 
that local effort up to but not beyond 10 percent 
of the state average would be rcrarded. 

Johns make a series of assumptions including 

that Aj° $600 and that foundation aid per I^* $400 

and computes an example if tlic 1965-66 funds wore 

distributed in accordance with his proposal. The 

data below are taken from his Tables 2 and 3. These 

data s1k)w: in Col. 1 three districts - the one with 

highest D^, the one closest to the state average, 

and the one with the lov/est D ; in Col. 2 the 

m 

respective D^; in Col. 3 the incentive grant; in 
Col. 4 the amount of minimum foundation program 
grant; in Col. 5 the total grant; and in Col. 6 the 
grant which wuld have been given to tlie district if 



Col. 1 


Col. 2 


Col. 3 


Col. 4 


District 


D 
ffl 






H 


1.452 


76,560 


46,400 


M 


.997 


369,688 


247,200 


L 


.262 


5,502 


14,000 
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all fimds wore distributed as foundation grants. 

Col. 5 Col. 6 
Total If all aid wero G| 
122,960 116,000 
616,888 618,000 
19,502 35,000 

Because of the upper bound (D^"^ 1.100) antl no lower 
bound, the program appears to have more of a penalty 
characteristic than a re^vard characteristic, 
e. Conceptual Framework Disaission. This program 
has the school district as the organizational target. 
The nature of the cliange that is sought is in 
structural mechanisms through which the local support 
for education is determined. The goal, procedure, 
results, and measurement are all in the input domain. 
A monetary reward, paid individually, through a change 
in competitive structure but involving the mechanism 
of a superordinate, is used. 
2. Meeting the needs of educationally deprived children 
a. Goal . Title I of the Rlemontary and Secondary 
Education Act of 1965 providetl Federal assistance 
to local education agencies to improve educational 
programs for children of low- income families. 
Section 101 of the act states (in part): "... The 
Congress hereby declares it to be the policy of the 
United States to provide financial assistance . . . to 
expand and improve their educatioail program ... to 
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meeting the special educational needs of educationally 
deprived children." (9) 

In 1970 Part B, providing for special incentive 
grants to be awarded, became effective. (30) the 
purpose for which money provided under special 
incentive grants was to be used is clear. (Section 122 
of Part B states (in part): "(2) (funds granted) usctV 
... to meet the special educational needs of educa- 
tionally deprivetl children." (9) 
b. Procedure . The procedure includes the determina- 
tion of individual state indices and a national index 
of effort. The indices of effort arc ratios (expressed 
in percents) of expenditures of all non-Federal money 
spent on public elementary and secondary education 
to total personal income. For the individual states 
the calculations are done on state expenditures and 
personal income within the state; the national index 
on the totals of all states. Tliose states whoso 
indices exceed the national index are entitled to a 
special incentive grant. 

The states, however, arc required to make applica- 
tion for the special incentive grants and to include 
appropriate assurances that the funds received will 
bo made available to local educational agencies and 
will be used to meet the educational needs of educa- 
tionally-depriver' children. 
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c, Measurement . Little or no assessment is required - 
the basic measures being the computation of the 
effort indices fnd the determination of the number 

of qualified children, \vhich is detemined in any 
event for the purpose of the basic grants to local 
educational agencies. 

d. Re^vard . Those states whose effort indices exceed 
the national effort index are entitled to receive a 
special incentive grant. The amount is dependent 
upon the extent to which the state index exceeds 

the national index and the number of low- income 
children in the state. A state may receive $1 . 00 
per child for each .01 percent by which the state 
index exceeds the national index. 

Note that, although the special incentive grants . 
are given to support compensatory education, the 
basis of rewrd is total alucational expenditures. 
This has been criticized, for example, by Wilcnsky 
(40) as an inappropriate use of incentives to promote 
compensatory education and also on the basis that 
individual districts can do little to contribute to 
the state index. 

e. Conceptual Framework Discussion . This program 
is targeted at the local educational agency (i. e., 
scliool district) but involves the state as an inter- 
mediary. The goal is in the process domain, but the 
results and mcasuranent are in the input domain. 



The monetary reward is paid on a group basis (to tho 
state for subseciuent distribution) through a competi- 
tive structure, but involving the actions of a 
superordinate. On tho basis of the disparity of 
the goals and results, this must be considered to 
be an inconsistent program. 
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II, INDIVIDUAL TARGKTS 

The incentive programs which hnve received the most attention are 
those vd\ich have been directed tou-ard the personnel in the educational system. 
These programs have included special contracts to test in' Dntives only as well 
as attempts to intro<]uce merit pay systems. Programs in the latter category 
have raised considerable controversy because they have emphasized achievement* 
usually academic, by the students as a basis o£ the re^v-ard. Using the input/ 
process/output matrix, however, it may be argued that a salary schedule in the 
more traditional vein based on both preparation and longevity is also an 
incentive program. In this section examples of incentive programs for school 
personnel are given. 

A. Input Domain 

A case reported by Chaplin (5) is used here as the first example of 
incentive programs for school personnel. 

X, Goal 

The basic goal is to promote continuing professional 
growth where professional grd\rth is, for the purposes of 
the plan, synonymous with additional academic preparation 
up to the MA + 18 units. 

In addition, a special aspect of the program 
(Project LEAP - Leadership in Hducation Advancement 
Program) provides for rewards beyond the last academic 
preparation step QV^ + 18). The decision to take and 
complete additional academic wrk is entirely at tlie 
discretion of the individual teachers. 

The salary schedule has two dimensions; (1) seven 
steps of academic accomplisliiment (Step 1 * R\ through 
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Stop 7 • M\ ♦ 18) and (2) longevity with one annual 
Increment at Step 1 and 10 annual increments at Step 7. 

2. Procedure 

The procedure is quite slr^plo. For the basic schedule 
the teacher takes additional academic wrlu 

For Project LFAP the teacher must complete a project 
vfhich enhances the etlucational program of the district 
(Hartford, Wise, Union High School). The projects must 
be approved in advance and ccmpleted satisfactorily. 

3. Measurement 

The academic aspects of the program are practically 
self -administer ingt with the teacher required only to 
present evidence of satisfactory completion of college- 
level course wrk. 

. For Project LEAP the administration must approve 
the project in advance and must reviev/ it for satisfactory 
completion. Project LEAP increments are given only for 
those projects which, in the judgment of the Administra- 
tion, enhance the educational program of the district. 

Completion of the proper nimber of academic credits 
moves a teacher from one step (for example, BA) to the 
next step (for example, M + 8 urilt?); the differential 

.:l||||he f irll'^lgillillrjl^ ^ : 
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carrying tho additional incentive of more salary Cor 
more longevity. On the other hand, there is a distinct 
penalty (an irmcdiato uj^per bound on salary) if a teacher 
elects not to continue with academic work. 

For Project LEAP (teachers at the highest step • 
^^ + 18) tho rewards range from $200 to $400 and become 
a permanent part of the teacher's pay. Only ono project 
may bo carried out each year, although there is no limit 
on the number of projects. 
S. Conceptual Framework Discussion 

This program has an individual target - the teacher. 
The goals, procedures, results, and measurement are all 
in the input domain. A monetary reward is paid individ- 
ually through a superordinate mechanism. 
B. Process Domain 

Thomas and KfcKinney (43) treat incentive or merit pay plans as 
one of several pos sible programs tliat are part of the accountabl 1 i ty movement . 
They point out that: 

The incentive pay plans advocated today seek to replace or 
. * supplement a salaiy schedule based on education and longevity 
with a plan that provides rewards to teachers commensurate 
with performance. (43, p. 40) • 
The usual incentive is a salary differential generally awarded 
for sui^erior performance in the standard teacher role. In this context 



Differentiated staffing generally provides not only for differential 
rolos for teachers but also for differential pay according to the rolo. (48) 
Within any one role, however, there is usually not a differential pay based on 
student achievement, which is, as noted above, supposed to be a prime factor 
of merit pay plans. lyiodos and Kaplan (32) inclulc descriptions of 11 merit 
pay plans, none of which seems to havo as a feature the measurement of student 
achievement as a prime or even partial indicator for determining reward. 

The following example, taken from Rhodes and Kaplan (32, p. 72-7S) 
although using differentiated titles does not apparently provide for differen- 
tiated roles and is more correctly termed a "merit-plan." 

1. Goal 

This plan provides for teachers to be placed on 
three schedules: (1) basic, (2) career placement, and 
(3) master placement. The purpose of using varying 
schedules is as Rhodes and Kaplan quote from the school 
district handbook to "recognize the quality of teacher 
training and exi^erience." 

2. Procedure 

Each of the three schedules provides for academic 
preparation (BA, Nt\, or equivalent, andMA + 30) and for 
up to IS years' experience. Tlie Basic Schedule is for 
new (probationary) teachers and tenure teachers who 
are evaluated as satisfactory. Tlie Career Placement 
Schedule is for probationary teachers with at Ibast 
4 years' experience (2 in the district) and for tenure _ 
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teachers. Transfer froif» the btisic to the Career Place- 
ment Schedule also requires a valid teaching certificate, 
professional gro^rth evidenced by at least 6 hours' (or 
equivalent) of graduate work, and evaluation rating of 
at least above satisfactory. 

The Master Placement Schedule is for tenure teachers 
or probationary with 5 years of experience (2 in the 
district). Also required are: (1) an MA and valid 
certificate, (2) professional growth, and (3) evaluation 
of outstanding. 

Note that although BA is sho^vn on three schedules, 
the requirements as stated above require an MA for the 
^fester Placement Schedule. 

Evaluation is done by the administrative and super- 
visory staff for all teachers as thny become eligible 
and each 2 years thereafter. Written evaluation forms 
are used and reviei^etl with the teachers , who have the 
right of appeal. Pinal decision as to schedule place* 
ment rests with the Board of Mucation. 
3. Measurement 

Teachers are evaluated on five factors! (1) personal 
fitness for teaching, (2) classroom effectiveness (the 
major criterion), (3) relationship with students, 
(4) relationship with parents, and (5) relationship 

'■■'■it- 

with staff. Each factor is rated on a five-point scales 
(1) outstanding, (2) excellent, (3) above satisfactory. 
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(4) satisfactory, (5) needs inprovcncnt. Bvaluation 
is done in accordance with a witten giiide. 
4. Reward 

Teachers who are placed on the Career Placement 
Schedule receive an initial increment of $600 and sub- 
sequent annual increments of $350, assuming they maintain 
an above satisfactory- to -excellent evaluation rating. 
Placement on the Career Schedule may occur at the fifth 
year. At tliat point the annual increment on the Basic 
Schedule is $300; hence plac«nent on career schedule 
carries twice the annual increment. Additionally, from 
the sixth year on the basic schedule has annual increments 
of $250 compared to the $350 annual incrment for the 
Career Placement Schedule. 

. Placement on the Master Schedule carries an initial 
increment of $800 and subsequent annual increment of 
$450. Placement on the ^^aster Schedule may occur at 
the sixth year, at which time the Basic Schedule has 
only a $250 increment. 

Thus, for example, three teachers each with an M/V 
would receive the following salaries at the sixth and 
tenth year: 

Year Basic Career Master 

6 8800 0200 0350 

10 0800 10,600 11,150 
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Tho. additional monies paid on the Career and 
Master Schedules are dependent on a continuing level 
of performance necessary to meet the requirements of 
the schedule. Teachers who fail to meet the eligibility 
requirements may receive only a partial or no increment, 
or may be returned to the Basic fichedule. 
5^ Conceptual Framework Discussion 

This program, like the last one, has an individual 
target - the teacher. Tho program has goal, results, 
procedures, and measurements in both the input and 
process domains. Hio monetary rewrd is paid Indi^ 
vidually through a sui^erordinatc mechanism. 
C. Output Domain 

There have been many recent programs, experimental in nature, 
which involved the use. of incentives to improve academic performance, ^tost 
of these programs had the student as the primary recipient. 

Some experimental programs either focused on .the teacher or 
included the teacher and sometimes the parent as a part of a more inclusive 
design. 

During the school year 1971-72 the U. S. Office of Education 
designed and implemented a 1-year project on the use of incentives in education. 
In general, the experiment involved four school districts. In each district 
there wore experimental and control schools. TV-o of the districts had 
incentives to teachers only programs and two of the districts had incentives 
to teachers and parents programs. 
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Planar Corp. (31) in Its iiiipact evaluation report generally 
concludes that the teacher only model did not demonstrate any differentially 
positive effects over the control school in improving student perfornanco. 
On the other hand, the teacher/parent model did seem to have such effects. 
The report notes, however, that tv^'o features of the design Impose severe 
limitations on the inferences which may bo dra^-n from the study. (31, IV - 
7 ^ 8) Tliese design features are the lack of randomization in assignment 
of treatments to units and the small number of experimental units (i.e., only 
two per condition) . 

The description which follows is based on the Planar Corp. 
report (31) cited above and Blaschke's report. (1) Tho tw models (teachers 
only and teacher/parent) were quite similar, hence described Jointly. 

1. Goal 

The goal of this project was to demonstrate tho 
feasibility of offering incentives to improve school 
learning. Two models of incentive were used: one, 
to teachers only; the other, jointly to teachers and 
parents. 

2. Procedure 

The procedure involved: 

(1) Four locations (medium-sized city districts), 
two for the teacher-only incentive itfodel and 
tw for the teacher-parent model 

(2) Within each district establishing experimental 
schools (tlK)Se offering incentives) and control 
schools (roughly comparable to exi^erimontal 
sc^odls) not offering incentives 
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(3) Using standardized achievcnent tests to measure . 
school (academic) learning 

(4) Establishing (for the academic improvement) a 
Basic Gain Indicator (BGI) - having an approxi- 
mate Grade Equivalency (GR) 

(5) AAninistering standardizetl tests at the beginning 
and end of the experlment^il period 

(6) Paying an incentive bonus based on the test 
score gain 

(7) Using other measures (for examples questioh- 
naires, interviews, school records and 
observation) to assess student, teacher, and 
parent bdiavlor and/or attitudes. Those 
types of measures, although used as partial 
indications to evaluate the project, had no 
bearing on the incentive payment. 

; For the purposes of measurement relative to incentive 
payment only the ^!etropolitan Achievement Test Batteries 
ifpl; were used. (27, p. 2-4) Students were given a test 

level suitable to their reading level as determined 
by their school records. A pre- test was given early 
in the school year (within the first mon^ of school) 
a!iy,,ij:^|||cy|t-y^^ : : 

:;iiMliWiiSKWMBMiliiillS^ : 
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4. Reward 

Tlie incentive to be paid depended on the average 
(mean) improvement of the class in terms of the basic 
gain indicator. The payments wore made as follows: 



Improvement Teachers Parents 

Loss than BGI None Nbno 

BGI $150 $12.50 

BGI + .1 GE 300 25.00 

BGI + .2 GE 450 37.50 

BGI + .3 GP, 600 50.00 



Teachers received the amount shown regardless of 
which model they took part in. 
5. Conceptual Framewrk Discussion 

This program had Individual targets, one portion 
having teachers as targets, the other portion having 
teachers and parents as the targets. The goals, results, 
and measures wore all in the output dcmin, the pro- 
cedures apparently in the process domain since the 
program did not deal explicitly with the question of 
how the improvanent in student porfonnance was to occur. 
The monetary reward was paid individually on the basis 
of a measurement method determined by a superordinate 
mechanism. 

In another sense the overall concept of the program 
was directed at modifying the input domain, in terms of 
examining the impact on teachers of a change in monetary 
reward. 
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Preface 



This proposal is being written in response to requests 
from the Office of Long Range Planning (OLRP) of the New 
York State Education Department (SED) • The proposed work 
covers two different areas: the continuation of program 
analyses and the continuation of program designs addressing 
problems of educational accountability. 

As a result of SED resource allocation decisions, OLRP 
has decided to concentrate on two v/ork efforts: an investi- 
gation of the effectiveness of alternative methods for 
awarding categorical funds to school districts; and the 
development of effectiveness measures for reading programs* 
Because these two tasks relate to different parts of OLRP's 
program and are very different from each other, they are 
discussed separately in Sections I and II. In Section III, 
a work plan for each task is proposed. Section IV describes 
Riverside Research Institute (RRI) , its capabilities, and 
its key personnel. 



RRIP/851-120/281 



Table of Contents 

Page 

I* A Program Analysis of Categorical Funding 1 

A, Problem Statement 1 

1, Situation 1 

2, Project Goal 4 

B, Approach 5 

1, Overview 5 

2, Phase I — Program Analysis 6 

3, Phase II — Program Design 15 

4« Summary 16 

II, Program Design Research: Effectiveness 

Measures in Reading and for School Operations 18 

A. Background 18 

B^ Proposed Effectiveness Measures 25 

1. Reading Effectiveness Measure (s) 25 

2. Orie^ntation Towards Learning: A 
Mea^^ure of the Effectiveness of 

School Operations 39 

3. Valuing Learning and School: A 
Measure of the Effectiveness of 

School Operations 43 



RRIP/851-120/281 



- iii 



Table of Contents (Cent MO 



Page 



III, Work Plan 49 

At A Program Analysis of Categorical 

Funding 49 

!• Project Goal 49 

2. Work Tasks 49 

3, Management Plan 51 

B. Program Design Research: Effectiveness 
Measures in Reading and for School 

Operations 52 

1, Project Goal and Objectives 52 

2, Work Tasks 52 

3, Management Plan 56 

Riverside Research Institute 

Social Systems Division: Key Personnel 58 




. RRIP/851-120/281 
- IV - 

FRir 



I. A Program Analysis of Categorical Funding 
A* Problem Statement 

1. Situation 

Categorical grant programs increasingly have become 

favored devices by which legislative bodies respond to demands 

that specific problems be addressed* Both the number of 

catogorical funding programs and the dollar amounts involved 

are growing rapidly. The number of Federal categorical funding 

programs to State and local governments has increased from 23 

in 1945 to an estimated 400 in 1966, The Federal dollar 

amount rose from $3.3 billion in 1956 to $20.2 billion in 1969, 

and is expected to rise to $30 billion by 1975. The figure 

for 1969 represents almost 22% of domestic Federal expenditures. 

State governments also make heavy use of categorical funding 

programs through grants to localit,\es--a total of $23.8 billion 

in 1969. 

The expansion of categorical funding programs has 
been nowhere more evident than in the education of the dis- 
advantaged. Two nearly identical programs / one State ^ under 
the Urban Education Act, and one Federal, under Title I of 
the Elementary and Secondary Education Act, together funded 
262.2 million dollars in 1971-72 for projects to benefit 
disadvantaged students in the local districts of New 
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York State. Still other programs have the same general 
goals and procedures** 

The fiscal consequences of these programs have been 
mixed at best. That the funds have been expended is clear, but 
whether these funds have been used as supplements to rather 
than substitutes for local revenues i or whether they have 
been concentrated on the target groups rather than being 
diffused to the general student population, are not at all 
clear. 

Measurable results in terms of students' growth are 
even less encouraging. A recent study by the American 
Institutes for Research in the Behavioral Sciences (AIR) is 
typical of numerous studies of the impact of categorically 
funded compensatory education programs. This analysis examined 
1,200 evaluations from various sources where it had been re- 
ported that a project had produced cognitive benefits that 
were statistically and educationally significant, AIR 
evaluated these projects and their evaluation reports against 
the criteria that, to be labeled "successful," a project would 
have to produce achievement gains for disadvantaged children 

^— — ^ 

E.g., Headstart, Follow Through, Bilingual Education, Drop- 
out Prevention, Adult Basic Education, Upward Bound, Education 
of the Handicapped. 

** 

Reported in Report on Education Research , Education Nev7s 
Service, October 17, 1971, pp. 6-8. 
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which were greater than those of their more advantaged counter- 
parts* Further, this rate of gain would have to be maintained 
until the disadvantaged children actually caught up# The 
successful project also had to include a representative sam- 
ple of not less than thirty children, and achievement gains 
had to be measured by some reliable testing instrument. After 
applying these criteria, only 326 of the original sample of 
1200 survived. Analysis of the remaining projects showed 
3.1 percent of these 326 were "successful in terms of the 
criterion describe^' above. 

The AIR study provides a bleak picture of the 
projects. Similar analyses have arrived at similar conclu- 
sions. 

The problem with these studies is that they provide 
few clues as to "why" a few projects succeed while most fail. 
That some succeed suggests that compensatory education is not 
impossible; that most fail suggests that something fundamental 
is wrong. What is wrong, and why it is wrong, are not ad- 
dressed. As a consequence, these studies supply little 
insight into what can be done at any level to increase the 
effectiveness of categorically funded compensatory education. 
Moreover, these "evaluation" studies encourage simplistic (but 
politically popular) conclusions that the problems of the dis- 
advantaged cannot be ameliorated through educational programs. 
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What is needed is a program analysis which system- 
atically considers alternatives for achieving specific goals 
in compensatory education^ This analysis should include 
alternative means of conducting categorical funding programs^ 
alternative supplements to categorical funding^ and alterna- 
tives to categorical funding itself. In New York State, the 
program analysis should concentrate on ESEA Title I and the 
Urban Education Act because, together, they account for many 
times the number of projects or the amount of funds involved 
in all other categorically funded compensatory education pro- 
grams combined* For the New York State Education Department 
(SED) , the program analysis should answer the following 
question: What should SED do to improve the effectiveness of 
Title I and Urban Education projects — and how should it be 
done? The proposed study is designed to answer this qii><3t;ion, 
to lead to well-reasoned recommendations for future ?£D pro- 
grams and to plans to implement the recommended pro^^amsi 
2. Project Goal 

To support the objectives of Title I and Urban 
Ed^ication, SED does not act directly bn disadvantaged 
students. Instead SED works through local education agencies 
(LEAs) which plan, manage, and evaluate the funded projects. 
Other things being equal, the effectiveness of SED's programs 
depends on the quality of the LEAs* performance. Hence, SED's 
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programs are undertaken for both immediate and ultimate pur- 
poses. The immediate purpuse is to improve the quality of 
project plmning/ management, and evaluation, functions cur- 
rently performed by LEAs. The ultimate purpose is to benefit 
the educationally disadvantaged. While the ultimate purpose 
obviously takes priority, fulfillment of the immediate pur- 
pose is a necessary intermediate step. The proposed project 
reflects this formulation in adopting as the goal for the 
project to determine how SEP can improve planning^ management ^ 
and evaluation of projects funded under Title I and Urban 
Education . The achievement of this goal will constitute a 
first step toward improving the education of the disadvantaged, 
and will provide a framework for recommendations concerning 
categorical funding as a means for achieving specific educa- 
tional goals. 
B. Approach 

1. Overview 

The proposed project pursues the goal of determining 
how SED can improve project planning, management, and evalua- 
tion. The project will be performed in two phases: 

Phase I - Program Analysis 

Phase II - Program Design 

Phase I will recommend what SED should do to im- 
prove planning, management and evaluation of projects funded 
under the Title I and Urban Education programs. Phase II will 
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reconunend how SED should carry out the recommendations of 
Phase !• The two phases will be sequential, but the approach 
within each will be iterative and as empirical as possible 
given extant data. 

2 . Ph ase I^-^'Program Analysis 

Four tasks will comprise Phase I. Task 1 will con- 
sist of a description and evaluation of SED^s present opera- 
tions in carrying out the Urban Education and Title I programs. 
Descriptive data will be drawn from an examination of extant 
documents and from interviews with SED personnel. This informa- 
tion will be reduced to functional descriptions (flowcharts 
and prose) of the present operations of the two programs. 

Two sorts of evaluation of present Title I and 
Urban Education programs will be conducted. First will be a 
process evaluation which will consist of a critique of the 
functional descriptions of the present SED programs against 
contemporary principles of planning, management, and evalua- 
tion. This evaluation may suggest near-term improvements in 
bed's processes. 

The second evaluation will assess the extent to 
which SED's present Title I and Urban Education programs are 
effective in bringing about the immediate objective of 
encouraging high quality planning, management, and evaluation 
of LEA projects. Data for this evaluation will be drawn from 
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a systematic examination of a sample of the project alloca- 
tions and evaluations submitted to SED by LEAs under the two 
programs. This second evaluation will provide a base-*line 
for comparisons of alternatives in Tasks 2 and 3, 

Task 2 will estimate the relative effectiveness of 
alternative incremental changes in SED's present programs , 
leading to a trade-off analysis among these options. The 
incremental changes that will be examined will consist of 
alternative combinations of critical design elements in the 
present programs, for example; the rigor of the requirements 
for project documentation {plans and evaluations) by LEAs, 
and whether funds are allocated by formula or by project 
competition; whether LEAs are expected to assume funding with- 
out SED support at some future date. 

The first approximation to these estimates will be 
drawn from a comparison of the effects on LEA performance on 
several of SED's present programs which differ with respect 
to these critical design elements. Data for this comparison 
will be drawn from a systematic examination of a sample of 
LEA applications and evaluations much like the examination 
in Task 1, but including categorically funded, compensatory 
programs other than Title I and Urban Education. For 

* This will also provide, as by-products, data for summary 
evaluations of the selected programs. Should these evalua- 
tions reveal that any of the comparison programs have been 
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example, BSEA Title I and Urban Education dififer principally 
in the rigor of documentation requirements? Bilingual Educa- 
tion is a project grant but Title 1 and Urban Education are 
formula grants; and so on. 

The first approximation will be refined by con- 
structing and using a crude description of the cause-effect 
relationships operative in project planning, management, and 
evaluation in LEAs. Correlations among LEA performance indica- 
tors and between LEA performance and such factors as district 
size, wealth, homogeneity, etc., will contribute to this 
description. Further refinement will be provided by a series 
Of on-site observations and interviews in a small number of 
LEAs selected as "extreme cases" of high and low quality planning 
and evaluations as determined in the examination of LEA docu- 
ments in the first approximation, described above. The field 
work will help Identify important differences between LEAs 
having high and low quality performance. The resultant des- 
cription, actually a very crude model will permit informed 
judgments as tr> why some projects fail while others succeed 



particularly effective in eliciting the desired performance 
by LEAs, detailed analyses of these effective programs could 
by undertaken within the proposed work effort (if mutually 
agreed to by SED and RRI) or as a separately supported effort. 
Such additional analyses, which could include process evalu- 
ation, would be directed toward determining the reasons for 
the superior performance. The results of the additional 
analyses could be folded into the recommendations concerning 
Title I and Urban Education, as well as into recommendations 
concerning other categorical funding programs. 
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and what dffeots on LEA performance can be expected from 
Incromental changes in SED* a programs. Specifically, the 
description will guide the decisions as to whether the i;e- 
suits of earlier comparisons can be extrapolated^ oil^ whether 
(and if SO/ how) they should be modified in making the pre- 
dictions for a trade-off analysis among the incremental 
alternatives. 

In Task 3 r the search for alternatives will be 
broadened to include non-incremental alternatives and a second 
trade-off study will be conducted, A preliminary set of non- 
incremental alternatives is presented below • The development 
of the description of LEA performance will augment and refine 
this list. This preliminary set of alternative SED programs , 
however/ excludes "strawman" design concepts as wasteful of 
resources both in empirical analyses and in subsequent trade- 
off analyses. 

(a) Upgrading the quality of staff performance 
within the LSAs might be indicated if the above analyses 
identified a lack of LEA personnel capability as a major 
source of ineffectiveness. Three particular capabilities are 
distinguishable s project planning, project execution and 
project evaluation. Either of the two sub-alternatives below 
might address any of these functions or any combination of 
them. 
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(a.l) Training of LEA personnel might hold 
promise If it were found that^ although LEAs do not presently 
have the needed capabilitiesi they contain people (e.g. r the 
present "coordinators") or can a-tract people who can perform 
these functions well. In this case, SBD might assist in the 
selection of LEA personnel and might conduct LEA personnel 
training programs at summer workshops/ on the job, at branches 
of the State University r etc. 

(a •2) Attaching SED personnel to LEAs might 
be indicated if LEAS appeared to lack (and to be unable to 
attract) persons who could conduct project planning/ execu- 
tion/ or evaluation/ but had staff members who seemed to be 
receptive to working with SED personnel skilled in these 
functions. SED personnel could work solely within one LEA/ 
or could serve several. They could interpret their roles as 
agents of SED providing consultative support to LEAS/ or they 
could interpret their rolos as members of the LEA staff. 

(b) Increasing the accountability of the LEA Boards 
and Superintendents might be a reasonable alternative if the 
data suggested that skilled LEA personnel either were or 
could be made available/ but that other constraints were 
impeding their performance and that increased visibility would 
improve their effectiveness. The two sub-alternatives differ 
in that the first increases accountability to the RegentS/ 
while the second increases accountability to the community. 
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(b.l) Increasing accountability to the Board 
of Regents as the state authority responsible for the educa- 
tion of the citizens of New York State, might be accomplished 
by increasing the rigor of the review processes conducted by 
SED, Such a step might prove premising if the analyses in 
T sk 2 suggested that projects funded under Urban Education 
were more effective than those funded under eSEA Title I. 
There aro many ways of increasing the rigor of review pro- 
cesses and further design work would be required to formulate 
concrete recommendations. Changes in resource allocation 
procedures might or might not be involved (see alternative 
(c) , below) . 

(b.2) Increasing accountability of LEAs to 
their communities might improve effectiveness if it appeared 
that Boards of Education, Superintendents/ and LEA staff mem- 
bers were highly motivated by community sentiment. SED's 
activities under this sub-alternative might take the form of 
formal audits of LEA proposals and evaluations. These audits 
would appraise the validity of the data, analyses, and inter- 
pretations of results contained in LEA evaluations, and would 
include a written report to the LEA Board. This report would 
affirm that the LEA staff 's procedures were valid, or, if 
they were not, would state why they were not, and provide 
appropriate, illustrative commentary. The Board and Super- 
intendent would be required to make public the LEA'S own 
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documents/ SBD's audit report/ and/ if they desire/ their 
own comments/ within a prescribed time period i 

(o) Changing the basis for resource allocation in 
the LEAS might be appropriate if the earlier analyses indi- 
cated that L|;As lack the needed personnel capabilities; that 
the political constraints associated with pre-allocation of 
funds according to formulas constituted a major limitation on 
SED^s ability to improve LEA performance; or that LEAs were 
using ESEA Title I and Urban Education funds to meet basic 
operating costs ♦ This alternative leads to attempts to 
modify the resource allocation processes in the LEAs by 
changing the basis for granting funds to LEAs and thereby 
influencing the LEAs' allocation of these funds to projects. 

(c. 1) Constraining the LSAs to selecting 
among proven programs might be effective if many LEAs appear 
to be using Title I and Urban Ed funds for projects that are 
highly comparable with respect to the problems they address / 
the objectives they pursue / and the design concepts they 
employ. Under this sub-alternative/ SED would develop and 
test "canned" projects for use in LEAs. LEA staff would 
select among proven projects, and the funds specifically 
allocated to the LEAs would be just sufficient to implement 
and operate the chosen projects. No funds would be available 
for projects other than those developed by SED/ SED might 
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contract with or collaborate with publishers/ LEA personnel/ 
and others in developing and testing "canned^' projects, 

(0.2) Mixing block and project grants to 
LEAS might be effective if the LEAs' problems and progranuning 
needs were found to be diverse , and if LEAs presently haver 
or can attract and train personnel possessing programming 
capabilities. Under this sub-alternativei unconstrained 
or partially constrained block grants would be awarded to 
LEAs to bring each LEA budget up to the minimum needed to 
offset basic operating costs. In addition to these block 
grants, sizeable project grants would be awarded to those 
LEAs able to demonstrate competence in project planning^ 
execution, and evaluation. The award of project grants 
could be made through rigorous review processes conducted by 
SED personnel or by special review panels convened by SED. 

(d) Combinations of the foregoing alternatives 
and sub-alternatives supply a large number of additional 
options. A few examples follow: 

(d.l) The increasing demands on LEA Boards 
and Superintendents generated by making them more accountable 

(alternative (b) ) could be met in part by upgrading the 
quality of LEA staff performance within the LEAs (alternative 

(a)) . 

(d.2) Upgrading the skills of persons respon- 
sible for project execution in the LEAs (alternative (a)) could 
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be a critical element in making the SED-generated projects 
(sub-alternative (av3)) effective at the LEA level. 

(d. 3) An important condition for implementing 
sub^-alternative (d* 2) might be to increase the responsibility 
of LEA Boards and Superintendents (alternative (b) ) . This 
combination would permit appraisal of the performance of LEA 
staff by comparative analysis, because the projects of proven 
effectiveness would come to the LEAs with records of results 
from previous field trials, and because comparisons across 
LEAs using the same projects during a given year could readily 
be generated. 

Because the non-incremental alternatives lie 
farther from SED's present programs, greater uncertainty is 
present in estimates of their consequences. The description 
of LEA processes/ developed in Task 2, however, will provide 
a preliminary basis for making the predictions needed for the 
trade-off analysis. 

The. results of the above three tasks of Phase I 
will be reviewed and consolidated in Task 4 into formal 
recommendations concerning the program SED should adopt as 
potentially the most effective in raising the quality of 
project planning, management, and evaluation of projects 
funded under Urban Education and Title I, subject to con- 
straints of costs and feasibility. 
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betorroininc^ what should be done is only the first 
step. The next is to determine how to do it. Phase II will 
address the second question, A formal review by SED will be 
conducted between the two phases to insure the relevance of 
the design work in Phase IX. The description of Phase II 
must be more tentative than the description of the first 
phase because Phase II depends on the results of the first . 
phase for its substance. 

Phase II will consist of five tasks. Task 1 will 
begin with the reduction of the most promising SED program al- 
ternative(s) , as determined by the review of Phase I, to its (their) 
component activities and will conclude with the design of a 
system for executing and managing these activities. 

Task 2 will consist of the development of a broad 
strategy for implementing the system designed in Task 1. It 
will focus on staffing, costs, time-phasing, likely contingen- 
cies, and how these contingencies may be dealt with. 

Task 3 will consist of a sensitivity analysis, a 
challenging of the assumptions, conclusions, and concepts 
incorporated into the system design and the implementation 
strategy. This analysis will serve to identify major uncer- 
tainties and risks so they may be dealt with appropriately. 
These uncertainties will be divided into those which prohibit 
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implementation and those which can be coped with through 
modifications in the program design^ the implementation 
strategy/ or the iterative operation of the program after 
implementation. 

Task 4 will be the generation of a program develop- 
ment plan which is more specific than the implementation 
strategy drafted in Task 2. It will contain detailed plans 
(tactics) for implementing those parts of the program about 
which there is low uncertainty and plans for reducing any 
uncertainties which might inhibit current implementation. 
Plans for reducing uncertainty might include field tests, 
further analysis, etc. 

Task 5 will consist of the drafting of any policy 
position papers deemed necessary to support the Commissioner 
and the Regents in taking action based on the results of the 
study. Examples of cases where such papers might be useful 
include the relaxation of constraints (e.g., laws) which 
inhibit SED's effectiveness, and the adaptation of the re- 
sults of Phase I to the more general fields of categorical 
funding programs and compensatory education. 
4. Summary 

The approach described above provides: 
(a) the analysis needed to determine what program 
SED should conduct to meet its immediate purpose of improving 
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the quality of planning, management, and evaluation of cate- 
gorically funded projects to aid the disadvantaged^ 

(b) the program design necessary to manage and 
execute this program; and 

(o) the plans and position papers needed to reduce 
any uncertainty regarding the program to tolerable levels 
and to Implement the program design In an orderly and effec- 
tive fashion. 
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II, Program Design Research ; Effectiveness 



Measures In Reading and for 
School Operations 

A* Background 

OLRPi in the course of its work on the development of an 
accountability model for education In the State of New York i 
has analyzed norm- referenced, standardized test methodology 
to determine whether measures based on this approach are ' 
useful in constructing an accountability model ♦The results 

it 

Of OLRP's analysis are as follows. 



The demand for accountability has focused an 
increasing amount of attention on ways of measur-* 
ing the outcomes of education. This concern can 
be reflected in the considerable amount of debate 
in the past several years over the types of in- 
struments that should be used to measure educa- 
tional outcomes. Much of the debate has centered 
on the efficacy of norm-referenced testing with 
respect to its use as a tool for educational pro- 
gramming and evaluation. 

The primary conclusion that has been reached 
as a result of the analysis is that while these 
tests may serve useful purposes (if we make sev- 
eral stringent assumptions with such use) I they 
cannot effectively contribute to any accountability 
system where the emphasis is placed on measuring 
student progress relative to educational treatment. 



Freeborne, G. Staff Paper on '.'Refining the Department's 
Policy on Racial Balance." Attachment B: Norm- referenced 
Testing. New York State Education Department, September 5, 1972. 
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This conclusion Is based almost exclusively upon 
methodological conslderationsv Thus, what Is 
being said Is that even quality norm-referenced 
tests, because of the way they are constructed, 
cannot serv3 as the foundation of an account- 
ability system nor as useful tools for educa-- 
tional evaluation. If two other considerations 
in addition to methodological considerations are 
included, namely the operational problems involved 
in constructing a quality norm-referenced test and 
the problems of interpretation of norm-referenced 
test results (particularly in the hands of persons 
not skilled in norm-referenced testing methodology) , 
the evidence tends to indicate that these tests do 
much more harm than good. 

To answei.' why these tests are not useful in 
any accountability framework , it is necessary to 
examine the major purposes of the tests, and given 
the purposes, examine how the tests are developed 
to achieve those purposes ♦ The expressed intent 
of norm-referenced tests is to differentiate 
among students . The entire methodological frame- 
work is based upon developing an instrument which 
provides for a maximum amount of differentiation 
between individual scores. Thus, the guiding 
principle is one of relative comparison on indi- 
vidual to individual . Any individual scores from 
a norm-referenced test make sense only in rela- 
tion to some other Individual scores. 

A second important principle is that you not 
only have differences in individual scores , but 
that you have wide variation in differences . Such 
wide variation is necessary If individual differ- 
ences are to be statistically defensible, l.e., 
that the differences are not a result of luck, 
chance, etc. and that the differences are wide 
enough so that there is some level of confidence 
in making relative comparisons • 

Given the concern for identifying individual 
differences and the need for wide variations in 
differences, a methodology has evolved over the 
past 50 years to achieve these ends. The meth- 
odology has established, systematic and empirically 
validated ways of devising Instruments to achieve 
differentiation and wide variability of individual 
scores. The emphasis has focused on constructing 
tests composed of a number of items of known char- 
acteristics. The most important characteristic of 
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an item/ if differentiation is to be achieved, is 
that some percentage of individuals will respond 
to the item incorrectly* If all (or mostly all) 
individuals answer an item correctly (or incor- 
rectly) I that item becomes virtually useless in 
differentiating among individuals. Thus the driv- 
ing force is toward generation of items that are 
neither *'too easy" nor ^'too hard" in order to 
achieve a reliable degree of differentiation • (It 
has been proven statistically that the greatest 
degree of differentiation with wide variation is 
achieved if all or mostly all items on a test are 
of average difficulty^ i.e., roughly half of the 
responses on each item are correct.) 

The way a typical norm-^referenced test is de-* 
veloped is that a number of items are generated 
from a specific content area, the items are pre- 
tested on a sample of students, and information on 
their responses is collected. Those items which 
differentiate best are most useful for achieving 
the purposes of the test. Thus, before a test 
item is included on the final version of the test, 
it is not necessarily known how the item differ- 
entiates, or why i t di f f e ren tla tes , but what is 
known is that it does differentiate . The high 
correlation between achievement test results with 
intelligence or aptitude test results have led 
an increasing number of analysts to conclude that 
standardized achievement tests are not as adequate 
a measure of educational outcomes as they are of 
general mental abilities • 

Norm-referenced testing instruments are jpoor 
choices for individual or program evaluation pre- 
cisely because of the emphasis on differentiation 
and relative comparison of students. With an em- 
phasis on differentiation, questions which all or 
most children can successfully answer are elimi- 
nated, or reduced to an absolute minimum. Thus, 
if knowledge bases of children expand (as could 
reasonably be expected over given time intervals), 
little or none of this would be perceived in an 
examination of norm- referenced testing results be- 
cause the tendency is to eliminate common base in- 
formation known by all children. The second, and 
even more critical aspect, is the masking of group 
growth because of the emphasis on relative com- 
parison. A child is progressing "normally" if he 
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maintains the same relative rank on test results* 
Group progress (meaning Improvement In scores) on 
the lower end of the test results only tends to 
raise average performance for all test results, 
and over time, moves the average higher • If the , 
effects of this phenomena are combined with the 
effects of eliminating questions which are ''too 
easyi" it becomes fairly obvious that measurement 
of individual or group progress, particularly for 
evaluation purposes, is virtually meaningless. 

A growth score is the difference between two 
individual scores either on the same test, equiva- 
lent forms of the same test, or comparable tests. 
Serious methodological problems exist in using any 
norm-referenced testing instrument for measuring 
growth or "progress," either on an individual or 
group basis. Testing experts have recognized the 
limitations in measuring growth with norm- referenced 
test methodology, and generally caution against the 
use of the tests for such purposes. The primary * 
problem relates to an individual score and the error 
of measurement associated with that score. In com- 
puting individual growth scores, the error of mea- 
surement of the growth score is the sum of the 
errors of measurement of the two individual scores. 
Thus if growth scores are to be obtained which are 
statistically meaningful and not subject to a large 
error of measurement, it is necessary that such 
difference scores be quite large. However , when 
it is considered how the test is developed, it would 
be highly unusual to expect such large score differ- 
entials. What has been found in practice is that 
growth scores derived from norm-referenced tests are 
characterized by their extremely low reliabilities. 

Significant methodological problems are also 
encountered in the development of norm-referenced 
tests with respect to introduction of cultural 
biases into the tests, and with respect to the ade- 
quacy of sampling the domain of knowledge to be 
tested. Cultural differences are normally asso- 
ciated with different language patterns and thought 
processes. To develop a standardized test which 
adequately takes into account such differences is 
impossible . What occurs is that the test is de- 
veloped and normed primarily on the characteristics 
of the predominant culture. Any interpretations of 
test results of individuals who are not part of the 
predominant culture are suspect. 
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Adequacy of sampling the domain of knowledge 
to be tested involves content and behavioral con** 
8iderations# It is not only important to test 
what a student knows (content) but also what he 
is able to do with what h^ knows (behavior) • The 
adequacy of sampling from a content domain is not 
as much a methodological problem as it is an opera- 
tional problem* Sampling of the behavioral domain ^ 
however/ is a methodological problem for norm- 
referenced tests because many of the important edu- 
cational outcomes desired are either not clearly 
specified or are not susceptible to measurement at 
this time • Thus ^ norm-referenced tests have no 
utility in assessing or evaluating the degree of 
creativity in individuals r interests , positive (or 
negative) attitudes, values, and appreciation of 
art, music, culture, etc*, all of which are impor- 
tant educational goals; Lacking the capability of 
measuring these variables, norm-referenced testing 
instruments are forced to rely heavily on content 
considerations* This becomes problematic if empha- 
sis on improvement of achievement scores comes at 
the expense of other important, non-*measurable 
(at this time) goals. 



It is apparent from the OLRP review that norm-referenced 
test methodology will not yield a good measure of what stu- 
dents have learned from a sequence of educational programs ♦ 
Furthermore, the methodology, as indicated, has limited 
utility in measuring attitudes and other noncognitive out- 
comes* . 

The conclusion that norm -referenced tests are not use- 
ful in any accountability framework does not mean that an 
appropriate set of accountability measures, or even an appro- 
priate set of objectives for such measures, can be specified 
at present. Objectives for accountability have not been 
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' defined for two reasons. First, no view or model of account*^ 
ability has ever been gensrally adopted. Second, the state- 
o£-the art in education does not permit a demonstration that 
meeting one sequence of instrumental objectives in grades k 
through 12 will enable students to meet desirable terminal 
objectives better than another sequence of instrumental ob- 
jectives. In fact, taking reading as an example, the present 
state-of-the art in education does not enable us to know what 
terminal objectives (e.g., the ability to read at a certain 
speed and with a specified level of comprehension) to set 
at all. 

As a consequence of such gaps in the state-of-the art, 
OLRP has decided to undertake R&D efforts which will yield 
a preliminary set of alternative accountability measures. 
Although present cost constraints limit the number and ex- 
tent of development of these preliminary measures, even a 
limited set will be useful to help to define precisely the 
standards which are to be met by educators. 

In the following sections, three different account- 
ability measures are proposed. In accord with OLRP aims, 
these measures are designed to address different aspects of 
accountability. The first, a reading effectiveness measure, 
is designed to focus on the outcome aspect of accountability, 
i.e., on measuring how well students can read in terms that 
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will have meaning to reasonably educated laymen. The second 
measure is designed to focus on an operational aspect of 
accountability by determining how well school districts carry 
out: T;he necessary steps to create situations that enhance 
learning by students* The third measure focuses on another 
operational aspect: the extent to which school districts 
have established an environment which students positively 
value* The latter two measures^ in effect, pose questions: 
should schools be accountable for communicating to students 
what is expected of them in order that they can profit from 
daily learning activities; and should schools be accountable 
for convincing students that schools provide a positively 
valued service to students? 

The three proposed measures were selected after exten- 
sive discussions with OLRP staff. These discussions led 
not only to the selection of the three measures from a set 
which included other candidates, but also to a mix of re- 
search tasks making up an efficient R&D program for RRI» 
For example, as will become clear in a subsequent discussion, 
the work on the reading effectiveness measure required inven- 
tiveness. Tasks requiring inventiveness are more difficult 
to program than straightforward developmental efforts, and 
progress is rarely proportional to the number of people 
assigned to such tasks. Therefore, rather than massing 



Q 24 - 

ERIC 



RRIP/851-120/281 



manpower on tasks requiring inventiveness in order to satisfy 
SED's wish to obtain one measure (e.g. ^ reading effective- 
ness) before another r it was OLRP^s judgment to select a set 
of measures and tasks to optimally harness RRI's R&D capa- 
bility. Furthermore/ OLRP decided that it was preferable 
to take three measures through to the design concept stage* 
during the proposed R&D program in order to provide OLRP with 
a basis for structuring SED decision-making processes, rather 
than to attempt to take one measure completely through all 
developmental stages. 

The proposed measures also have secondary requirements 
which do not directly relate to educational accountability. 
These requirements will be discussed as the approach to the 
development of each proposed measure is reviewed. They have 
been included to improve the utility of the measures in 
meeting the immediate needs of managers in education. 
B . Proposed Effectiveness Measures 

1. Reading Effectiveness Measure (s) 

(a) Background and functional requirements . Begin- 
ning with the Coleman report, a number of recent works have 
concluded that education (as characterized by input variables) 



The meaning of "design concept stage" for each measure will 
be made clear siibsequently in this proposal. 
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is not strongly related to outcomes (e.g.^ reading achieve- 
ment) if the effects of such variables as socioeconomic class 
and IQ are taken into account or controlled. Yet the authors 
of these works acknowledge that learning how to read and how 
to perform basic arithmetic operations are among the capa- 
bilities largely acquired from educational programs • Further- 
more /they also acknowledge that mastery of some subjects 
such as arithmetic must take place predominately in the 
schools. Therefore it is important to understand why input- 
output studies have not shown that educational processes 
result in cognitive and noncognitive development. 

Part of the problem may lie in the fact that 
input and output measures are frequently correlated with 
one another because they are constructed from a common ^ norm- 
referenced methodology. For example, both IQ and standard 
reading tests share a common test-construction methodology. 
Therefore/ the scores of thet^ tii^sts may be highly corre- 
lated not because the phenomena these tests seek to measure 
are empirically related, but because both kinds of tests are 
actually (although unintentionally) measuring the same or 
nearly the same capabilities or capacities. 

If the method correlation between measures of 
input and output variables is high, it follows that a limited 
amount of variance remains to be accounted for by other 



iS 26 - 



RRIP/851-120/281 



variables, e«g»i educational programs. For example, after 
IQ and reading achievement scores are correlated/ there is 
a limited amount of variance left. If this limited amount 
of variance is too small to attribute to variables such as 
school programs, then it is no surprise that many studies 
have concluded that school programs do not strongly affect 
educational outcomes. 

There may be other reasons why norm-referenced 
tests are inadequate measures of educational accountability. 
Let us assume that ten years of formal education raises the 
reading abilities of the members of a group by some measur- 
able amount. Norm-referenced tests probably would never 
detect this change in ability. Because the tests are peri- 
odically renormed, the worst-performing student always de- 
fines the worst score. Therefore, the distribution of 
scores always looks the same, even though the worst student 
might be able to read a tax form after ten years of educa- 
tion while the worst student before ten years of education 
might barely be able to read traffic signs. Thus norm- 
referenced tests cannot reveal how much better or worse off 
people are today by comparison with any earlier time. And 
yet, it is well known that in the United States today, the 
level of literacy is substantially higher than it was in 
the 1920s. 
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The above discussion strongly suggests that 
the State needs new and different measures of effectiveness 
if it is going to account to the public for its activities. 
In the area of reading effectiveness, the new measure must 
satisfy the following functional requireraents: 

(a.l) It must determine how well individuals 
can read relative to the standards that define adult reading 
competence; 

(a»2) It must be able to measure sufficiently 
fine distinctions in reading achievement to allow for the de- 
termination of student progress from grade 2 to grade 12. 
(This contrasts with available literacy measures which are 
too crude for monitoring the performance of students in the 
primary grades;) 

(a. 3) If possible, the measure should enable 
the SED to set minimum quantitative and qualitative objec- 
tives (standards) so that school sysbems can be held account- 
able; 

(a. 4) If possible, the measure should increase 
the validity of decisions affecting manpower resource allo- 
cation and the value of cost-benefit and other input-output 
studied, whether they are executed for policy-related or 
operational reasons. 
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(b) Alternative ap^proaohes for developing standarcia 
of reading competence * 

(b*i) Expeotatlona defining adult reading 
competence . In order to develop a reading effectiveness 
measurer the terminal reading capabilities expected must 
be defined. There are several approaches which could be 
used to formulate such standards; 

• Specify a population of adult written 
materials and sample from this population a set which is 
then subjected to analysis andi ultimately , suomltted to a 
panel of judges to define the required standards. The stan- 
dards could be stated in the following form: "if certain 
specified conditions are met, an adult should be able to 
exhibit certain specified behaviors"; 

• Use any of a variety of methods (e.g. i 
the Delphi Techniques*) to produce a consensus on the 
standards; 

• Investigate theoretical models of 
language to see if they can be successfully used for the 
purposes of defining the standards. 



Weaver, T.W. Developing an accountable consensus of goals: 
The Delphi Technique. In Browder, L.H., Jr. (Ed.) Emerging 
problems of administrative accountability . Berkeley! Mccutchen 
Corp., 197i. 
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The National Assessment Program app*>ars 
to have elected the first alternative^ Ttie approach is ex- 
pensive! somewhat arbitrary (experts with established points 
of view define the standards) , the open-ended (language 
changes as society changes, and therefore it is necessary 
to re-examine standards periodically) . 

Problems are also associated with the 
second approach. Attempts to reach a concensus on any goal 
in education have produced useful results only when methods 
for lowering the level of abstraction in discourse have been 
introduced* Thus, for example, if the work required under 
the first alternative were accomplished, and if the people 
of New York State were given an opportunity to study the 
results, then something might be expected from using time- 

if 

consuming and expensive consensus-producing processes* 

The last alternative has never (to RRI's 
knowledge) been tried* Sufficient information exists con- 
cerning at least some aspects of language to recommend that 
formal models be examined to determine if they provide 



Although RRI does not recommend holding consensus-validating 
sessions throughout the State of New York, the people in the 
end have the prerogative to set and validate standards* It 
would be desirable to use methods other than consensus- 
contriving techniques for this purpose. 
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enough information to enable the formulation of the required 
standards of measurement. This approach would anchor per- 
formance standards to hard data. In addition ^ the third 
alternative has other advantages. First, j.t is probably 
the least costly appro&ch. Second, the approach may yield 
estimation methods to enable linguistic changes to be fore- 
casted. Third, and most important, the approach holds the 
promise of empirically testing assumptions concerning the 
importance of different sequences of expectations set for 
grades 2 through 12 to the achievement of adult competence 



IE ^ — — — — — " ■ 

All language has a history* The literat^i^^ shows that 
some aspects of language change more rapidly than others. 
Some of these changes are clearly of limited Interest in 
setting expectations. For example, words comi and go in 
English (as in every other language) as a function of our 
major preoccupations. The word "f lack" for example was 
introduced during World War II as an abbreviation of a 
German expression for an antiaircraft gun firing fragmen- 
tation shells. Many other archaic terms are with us still, 
and other words have long since passed out of common En- 
glish use. 

Some aspects of language change more slowly. It is pre- 
cisely these aspects, however, which should be the subject 
of projections. More specifically, we need to determihe 
whether materials written for the ordinary citizen have 
increased or decreased in "readability" over the last 
thirty years, and to measure the rate of increase or de- 
crease in "readability." This work may lead to assessments 
of what the adult world will expect some fifteen years 
from now of a five-year old currently entering kindergar- 
ten. On the hypothesis that we work out methods for mea- 
suring the "readability" of written materials, then it 
should be possible to make such projections. 
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in reading** This last application of measures of reading 
effectiveness is discussed further in the next section. 

(b.2) Setting instrumental objectives * A mea- 
sure of reading effectiveness must determine not only the 
extent to which an individual has achieved adult competence, 
but also the extent to which a student, at a given develop- 
mental stage, is progressing toward the expectations set by 
adults in the socialization process. Any of the alternative 
approaches outlined in (b.l) could be used to set the stan-* 
dards for each grade. However, the state-of-the art is such 
that no expert can be sure that, by acquiring capabilities 
X^, Xji . . in Grade 2, capabilities Y^^, Yg, . . ., 

in Grade 3, and so on, a student will eventually acquire 
adult competence in reading. The state-of-the art does not 
even permit the estimation of the probability of arriving at 
any level of reading competence if specified steps are 
followed. 

Any consensus reached by adults using 
the Delphi or any other technique (even if it were effi- 
ciently arrived at for each grade) would simply be a 



This work is beyond the scope of the proposed R&D research 
but it is an important component of a larger effort leading 
to reading curriculum validation. 
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consensus which could be worthwhile or worthless depending 
upon how well adults from all walks of life intuitively 
understand human development. The third alternative, while 
it does not yield direct knowledge of the standards which 
should be set in grades 2 through 12, has the prospect of 
at least describing present expectations/ and, in addition, 
of setting the stage for future research to show how the 
acquisition of reading skills can be simplified. 

The discussion of alternative approaches, 
both with respect to setting expectations (standards) of 
adult competence as discussed in subsection (b.l), and with 
respect to setting instrumental objectives for each of the 
primary and secondary grades as discussed in subsection 
(b«2), suggests that the third approach--the use of formal 
models of language to develop the standards of reading com- 
petence at all levels — should be pursued. 

c. Method of procedure . Discussions were held 
with OLRP concerning the trade-offs among alternative ap- 
proaches to developing standards of adult competence in 
reading, and the problems associated with setting standards 
for each of the grades 2 through 12. OLRP accepted RRiVs 
recommendation that an R&D instrumentation effort be under- 
taken to determine if formal models of language could pro- 
vide a basis for developing both standards of reading 
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competonce and new reading effectiveness measures. 

On the basis of an incomplete literature 
search, it appears that an experimentally confirmed mathe- 
matical (lognormal) model of words (tokens) and word types 
has been formulated for the written, adult language. Further- 
more, Carroll et al . * have shown that the tokens and types 
contained in written materials used in grades 3 through 9 
also can be described by lognormal distributions. 

If the lognormal model adequately describes 
the distribution of tokens and tynes in written English, 
then a quantitative description of writteii materials in 
terms of their level (i.e., grades 3 through 12 and adult) 
is at hand. Furthermore, it should be possible to quantify 
the relationships among the written materials used in dif*^ 
ferent grades and adult written materials. Finally, pro- 
vided that samples of words are properly drawn, and provided 
that students know the meanings of these words, it should be 
possible to assess quantitatively the extent to which a stu- 
dent has progressed toward mastering adult vocabulary in 
each of the grades in which reading is taught. 

The existence of such a model raises many 
possibilities for the identification of instrumental 



Carroll, J.B., Davies, P. & Richman, B. Word frequency 
book. New York: Houghton Mifflin, 1971. 
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objectives for reading programs. With such a model/ samples 
of written text used in various grades cpuld be analyzed and 
significant differences identified. Prom these differences^ 
the instrumental objectives which are currently governing 
instruction in each grade could be inferred. Suppose, for 
example, that analysis showed that, relative to adult English 
text, the most common words appeared more frequently in the 
lower than higher grades, that moderately common words ap- 
peared less often in the lower grades than in the middle 
or upper grades, and that the rarest words appeared only 
in the upper grades. Prom this information, a variety of 
instrumental objectives could be inferred: concentrate 
almost exclusively on teaching common words in the earliest 
grades; teach moderately common words in the middle grades 
and reduce the emphasis on reading the most common words; 
and introduce rare English words in the upper grades while 
sustaining, but at a reduced priority level, an interest 
in teaching the common and moderately common words. 

Once the instrumental objectives have been 
identified, it should be possible to measure the extent to 
which students have met the expectations embodied in the 
materials used in the various grades. Furthermore, if 
samples of adult written materials (newspapers, tax forms, 
etc.) were related to the model describing the full adult 
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written language, then a rational basis for defining the 
capabilities for competent adult perforinance in reading 
should be feasible « 

On the hypothesis that the R6D program is 
successful and RRI is able to properly sample word types 
for different purposes (to measure whether the expectations 
on students in each grade are being fulfilled and to mea- 
sure student progress towards becoming a reader of adult- 
level text) , it should be possible to devise test score 
reports that are understandable to parents as well as to 
other adults ♦ For example, we might be able to report 
that student X has learned the meaning of nearly all of the 
most common English words, most of the next most common 
words, etc* and, in the aggregate, has acquired Z% of adult 
competence* Since Carroll has shown that adults can ac- 
curately estimate the relative frequency of English words, 
such scores should be intuitively meaningful to parents 
and others. This procedure would represent an advance 
over the present practice of reporting test results in 
technical jargon, e.g., national or r^sgional grade norms, 
which many educators, no less laymen, don't understand. 



Carroll, J*B* Measurement properties of subjective inagni- 
tude estimates of word frequency . Princeton, N.J.: Educa- 
tional Testing Service, 1971., 
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Formally describing the distribution of types 
and tokens in English may not describe English sufficiently 
well for our purposes* Undoubtedly educators would want to 
test to determine whether students comprehended passages of 
English text. Adults expect that children will improve 
their grasp of the meaning of written materials as they 
grow. More particularly, adults expect that learners will 
understand more complex explanations as the learners develop* 
Furthermore, as learners grow, they are expected to acquire 
the ability to infer the intent of a writer from his selec- 
tive use of linguistic terms and forms, to be sensitive to 
alternative inferences which may be drawn, and to be aware 
of the variety of emotional effects which the writing might 
have on other readers. 

There is no known way to completely describe 
all the meanings of written materials. However, all that 
is needed at present is the capability to scale the diffi- 
culty, or so-called "readability," of passages of text. 
Such scaling is feasible. 

It should be i:easible to characterize the 
"readability" of materials used at different grade levels 
as well as samples of adult materials. If a passage of 
English text were given a score on a "readability" scale, 
then it should be possible to draw samples of English text 
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of known "readability" for the purposes of testing compre- 
hension. For example, if & student scores well on compre- 
hension for the most "readable*! text, less well for the 
next most "readable" text, and so on, then it should be 
possible to give the student an interpretable score. Further- 
more, since one of the criteria used in the literature for 
scaling the "readability" of English text is type and token 
content, it may be feasible to translate scores on reading 
comprehension into mastery scores based upon the signifi- 
cance of knowing the meanings of different words with dif- 
ferent probabilities of occurrence. 

Syntax is another aspect of language which 
might have to be taken into account in formulating a read- 
ing effectiveness measure. It is well known that, as 
children grow, they are able to understand increasingly 
complex syntactical forms. At this time, however, there 
are many issues to be resolved concerning the best method 
for precisely describing the syntactical complexity of 
English text. Certainly a variety of linguistic character- 
istics (e,g., sentence length) could be measured. Whether 
any such indicators, or some combination of them, can ade- 
quately describe the increasingly complex set of expecta- 
tions imposed on learners progressing from grades k through 
12, is an open question. In addition, at this stage of 
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the research, much uncertainty surrounds the need to develop 
a separate scale of syntactical complexity or a separate 
test for mastery of syntax. A review of the literature 
might show that syntax is co hopelessly confounded with 
vocabulary and with "readability" that a separate test for 
mastery of syntax would be unnecessary to obtain valid read- 
ing effectiveness scores. Indeed, since the various aspects 
of language seem to be interrelated, it remains to be seen 
whether more than one aspect needs to be sampled for mastery 
in order to formulate an adequate effectiveness measure. On 
the other hand, people might be more inclined to accept test 
results if they felt that all aspects of language mastery 
were tested. Furthermore, having several independent ways 
of estimating progress towards becoming a competent adult 
has its advantages from a construct validity point of view. 
2 . Orientation Towards Learning: A Measure of t| ie 
Effectiveness of School Operations ^ 
(a) Background . In colleges, formal courses and 
informal settings bringing together students and faculty 
are extensively used in an attempt to orient students to 
life in the institution and to the content and boundaries 
of various academic disciplines. In any educational environ* 
ment, attempts are made to structure all learning situations 
by using some means of communication (usually speech and 
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written text) to explain what is expected of students, both 
with respect to their learning activities (calculating, read- 
ing, role-playing, etc.) and with respect to the aims, or 
expected outcomes of these activities • The capability for 
setting up successful learning situations depends on the 
ability of teachers to communicate instructions to studcnti^^ 
Since this ability is an invariant expectation on teachers 
over the entire student age range and over all modes of in- 
struction, OLRP and RRI have considered the development of 
a measure to determine the extent to which this expectation 
is being fulfilled, especially in the primary grades. OLRP 
has decided, as indicated previously, that it would be de- 
sirable to develop a prototype measure in order to be able 
to put the question in less abstract form to SED decision- 
makers. Therefore, RRI has agreed to propose the develop- 
ment of such a prototype measure. 

(b) Method of procedure . One way of determining 
whether teachers are successful in setting up learning ac- 
tivities is to test whether students understand key instruc- 
tions, for example, the concept "given." An inderstanding 
of such key concepts by students is a critical pre-condition 
for successfully setting up learning situations. Differen- 
tiating between one instruction, "read," and another in- 
struction, "write," is also important to the success of 



- 40 - 



RRIP/851-120/281 



learning activities. Finally, insuring that all students 
share the meaning of key concepts and instructions is im- 
portant for two reasons. First, instruction in school situ- 
ations are most often given to a group of students, and 
second, there is a heavy reliance in educational institu- 
tions on students orienting each other so that group learn- 
ing is facilitated. 

A prototype measure may be formulated by sam- 
pling a set of key concepts related to the transmission of 
instructions to students in certain grades (say 2 through 
12). If a synonym for each key concept in the sample were 
included in a set, then students could be asked to judge 
the degree of similarity between all pairs of concepts in 
the set. If a particular pair of words meant the same 
thing, the similarity judgment should be at its highest 
value. If the pair of words had little similarity, the 
similarity judgments should be minimal. 

Analyses of the judgments should permit a 
determination of the degree to which students comprehend 
important concepts (i.e., can identify the synonyms) and 
understand significant differences among concepts. In 



It is likely that the form of the test and the instructions 
will be based on the People Test (see '^Guidelines for the 
evaluation of desegregation programs in school districts,*' 
RRI, September, 1972. 
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addition, it should be possible to derive a measure of the 
extent to which each student shares his similarity judgments 
with every other student • RRI believes that recent develop- 
ments in multi-dimensional scaling techniques may be useful 
in developing the required measure, 

(c) Other applications , llie proposed measure has 
several other possible applications. First r it could be em- 
ployed as a moderator variable in policy-related research. 
For example, if class size is reduced, some teachers may be 
better able to get students to understand and share 
instruction-related concepts than other teachers. This 
phenomenon might explain the weak relationships reported 
in the literature between class size and the cognitive out- 
comes of educational programs. Thus, the measure could be 
used (along with others) in investigations designed to de- 
termine optimal class size under different circumstances, 
e,g,, different classroom mixes of racial and socioeconomic 
heterogeneity. Second, the measure might prove useful in 
research directed toward understanding why some teachers 
are more effective than others, (For example, good teachers 
may spend a good deal more time early in the semester get- 
ting students oriented towards what is expected of them.) 
Third, the measure could be useful in accounting for ob- 
served differential effectiveness among educational programs 
designed to meet the same objectives. 



- 42 - 



RRIP/851-120/i81 



i 



3. Valuing Learning and School: A Measure of the 
Effectiveness of School Operations 
(^) Background s An Invariant expectation on all 
educators is that they should structure an environment which 
is positively valued by students. The rationale for this 
expectation is based both on research findings and on obser- 
vations made over the sweep of Western Civilization, Among 
the research findings are those which demonstrate a causal 
link between achievement and liking a teacher, or a field 
of study, or school in general. Among the axioms of educa- 
tors are such statements as: "If rapport is not established 
between learners and teachers, little will happen of any 
value"; or "Learning induces anxiety since we have to ven- 
ture and make mistakes. If a student believes that he will 
be negatively valued for making mistakes, he is apt not to 
volunteer to learn. Hence he will, to maintain self esteem, 
reject or ignore education," Therefore OLRP, as indicated 
previously, has requested that RRI develop a prototypical 
measure of student attitudes towards educational environ- 
ments. Again, the rationale is to develop a measure which 
could be used by decision-makers as another measure of 
accountability. 

(b) Method of procedure , RRI proposes to 
further develop its Test of the Special Meanings of Words, 
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which was designed and field- tested in previous work per 
formed for SED. A description of this test follows.^ 



Test format and administration > Students 
(grades 5-12) are asked to rate each concept 
against a set of bipolar adjectives. One set of 
eleven adjective scales is used for rating human/ 
animate concepts and another set of eleven for 
abstract/nonanimate concepts. Adjectives of ob- 
vious social desirability are excluded from the 
animate adjective group and descriptive or clearly 
denotative adjectives are excluded from the 
abstract/nonanimate group* The difficulty of the 
vocabulary is controlled so that all words in the 
test should be familiar to most upper third 
graders. Concepts and adjectives are either A 
or AA words in Thorndike-Lorge word book,* or 
appear on the Dale-Chall** or Stone *** easy 
word listB. 

Since grade school children appear to work 
better and with greater sensitivity on a five- 
point scale,**** such a scale has been used in 
the Test of the Special Meanings of Words rather 
than the morei frequently used seven-point scale. 
The center of the scale constitutes a neutral 



" "Effectiveness of racial balance programs; Pinal report 

for Phase IIA, volume I," RRI| September, 1971. 

* 

Thorndike^ E. & Lorge, I. The teacher's word book of 
30,000 words . Hew York: Teachers College Press, 1944. 

Dale, E. & Chall, J. A formula for predicting read- 
ability: Instructions. Educational Research Bulletin , 1948, 
27(2), 11-28. ""^ 

Stone, C. Measuring difficulty of primary reading ma- 
terial: A constructive criticism of Spache's measure. Ele - 
mentary School Journal , 1956, 51^, 36-41. 

ikit-kit 

Osgood, C.E., Suci, G.J., & Tannenbaum, P.H. The mea - 
surement of meaning . Urbana: University of Illinois Press, 

mi. — 
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poattton/ and the ends constitute the polar ex- 
tremes of the adjective pair. All scale points 
are clearly labelled (anchored) as to meaning, 
following the suggestion of McNeil and Phillips.* 
Students are asked to decide which of the five 
scale points of the adjective pair comes closest 
to describing how the concept being rated "feels" 
to them, and to place a checkmark at that point 
along the adjective scale. 

Factor analysis . Factor analytic studies of 
the Test of the Special Meanings of Words were car- 
ried out to determine whether factor structures 
were sufficiently comparable for black and white 
students at different levels of racial heteroge- 
neity to permit valid comparisons between groups. 

Scales by students matrices were obtained by 
summing over concepts. A separate matrix was ob- 
tained for black and for white students at each of 
four levels of racial heterogeneity. Each matrix 
was then intercorrelated to produce eight symmetric 
intercorrelation matrices, A principal components 
factor analysis was performed on each correlation 
matrix. The resulting factors were rotated to 
simple structure using a varimax rotation scheme. 
The factor structures were then compared. Since 
the first three factors accounted for over 80% of 
the common variance, the first three factors were> 
used for comparisons between groups. Coefficients 
of congruence** were calculated to compare the fac- 
tor structures of the eight groups. The coeffi- 
cients were all exceedingly high, indicating that 
the groups used sufficiently similar frames of 
reference in rating the concepts to allow compari- 
sons between blacks and whites and between stu- 
dents in classes of different racial compositions. 

Validity studies , There are many studies in 
the research literature which document the 



McNeil, K,A. & Phillips, B,N. Scholastic nature of re- 
sponses to the environment in selected subcultures. Journal 
of Educational Psychology , 1969, 60(2), 79-85, 

** 

Wrigley, C, & Neuhaus, J,0, The matching of two sets of 
factors, American Psychologist , 1955, 10, 418-419, 
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predictive and construct validity of the semantic 
differential (SD) technique {e.g*, Osgood et al.* 
and Reeves**). Moreover, it has been demonstrated 
that the evaluative dimension of the SD is an atti- 
tude measure, e.g., Likert, Thurstone, and Guttman 
attitude scales are highly correlated with the 
evaluative factor of the SD. Thus semantic differ- 
ential data include attitude as a component of the 
"meaning" they index. 

Since the validity of semantic differential 
techniques as attitude measures has been well docu- 
mented, construct validation analyses fpr the Test 
of the Special Meanings of Words have been limited 
to demonstrating that the four subtests of the in- 
strument measure the psychological constructs they 
were designed to assess. Thus validation analyses 
have been designed to show that indices derived 
from selected concepts differentiate between groups 
believed to differ with respect to sense of fate con- 
trol, social class, self-concept and attitudes to- 
wards school. Preliminary studies comparing white 
and black students in the test-bed sample are re- 
ported in this section. These black-white compari- 
sons make use of the statistic D, a generalized 
distance function which is an index of differences 
in connotative meaning between concepts.*** For 
each student who took the Test of the Special Mean- 
ings of Words, a D score was calculated for each 
of several key concept pairs. Analyses were then 
carried out to compare the D scores of white stu- 
dents with the D scores of E"lack students. 



Osgood, C.E., Suci, G»J. & Tannenbaum, P.H. The mea- 
surement of meaning . Urbana: University of Illinois Press, 

** Reeves, M.P. An application of the semantic differential 
to the Thematic Aperception Test material. Unpublished doc- 
toral dissertation, University of Illinois, 1954. 

*** cronbach, L.J. & Gleser, G.C. Assessing similarity be- 
tween profiles. Psychological Bulletin , 1953, 50, 456-473. 

Osgood, C.E. & Suci, G.J. A measure of relation deter- 
mined by both mean difference and profile information. Psy - 
ch ological Bulletin , 1952, 49, 251-262. 

— — Osgood, C.E., Suci, G.J. & Tannenbaum, P.H. The measure- 
ment of meaning . Urbana: University of Illinois Press, 
1957. 
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The Test of the Special Meanings of Words was 
originally developed to test hypotheses concerning some of 
the effects of school desegregation. (Other purposes are 
indicated in RRI^s report. The instrup^ent proved to be ex- 
tremely sensitive I see pp. 50-51 of the report.) 

Analyses of the Test of the Special Meanings 
of Words provides a basis for developing an efficient and 
more polished measure of attitude, in which the semantiq 
differential technique will be used to measure the extent 
to which students share and positively value school-related 
concepts. Thus , it should fulfill the requirements of an 
accountability measure. 

(c) Other applications . In RRI^s report, "Effec- 
tiveness of racial balance programs: Final report for Phase 
IIA, volume I," several other applications of the instru- 
ments are described. In the work proposed here, one addi- 
tional application will be attempted simply because the pro- 
posed accountability instrument can be constructed effi- 
ciently to meet an additional objective. 

RRI proposes to develop a score (perhaps a 
score based upon the D statistic) to identify students who 
do not share their evaluations of components of school en- 
vironments with others. RRI will develop the theory of 
such a measure for potential use in calling to th^ attention 
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of school administrators those students who negatively value 
school concepts. Such students might deviate sufficiently 
from others to be candidates for special courses of action. 
For example, the measure might identify candidates for a 
drop-out prevention program. Considering the number of 
drop-outs in urban areas (in New York City, 60% of the Span- 
ish surnamed students eventually drop-out, and some 50% of 
the black students drop-out) , it might be very worthwhile 
to detect potential drop-outs so that something could be 
done for them before rather than after they actually leave 
school. 



- 48 - 



RRIP/851-120/281 



Ill, Work Plan 
This work plan consists of two parts* Part A contains 
tasks which follow from Section I of this proposal. Part B 
contains tasks which follow from Section II of this proposal, 
A. A Program Analysis of Categorical Funding 

1. The goal of this effort is to determine how SED can 
improve planning^ management^ and evaluation of projects 
funded under Title I and Urban Education, The objectives of 
the two phases which will fulfill this goal are as follows: 

Phase I ; To conduct the program analyses 
necessary to recommend to SED the program which holds the 
greatest promise for improving planning, management, and 
evaluation of categorically funded projects to aid the disad- 
vantaged. 

(b) Phase lH To prepare the designs, plans, and 
position papers that will be necessary for the orderly and 
effective implementation by SED of the recommended programs. 

2. Work Tasks 

Riverside Research Institute (RRI) will apply its 
best efforts to carry out the Phase I and Phase II tasks 
enumerated below in conjunction with staff from the Office of 
Long Range Planning. 
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(a) Phase I » Program Analysis . 

Task (A.l) » A description and evaluation of 
SED'a present operations under ESEA Title I and the New York 
State Urban Education Act. 

Task (A. 2) : A trade-off analysis among incre- 
mentally diffosrent alternatives to SED's present Title I and 
Urban Educa tic n programs. 

Task (A. 3) > A trade-off analysis among non- 
incrementally different alternatives to SED's present Title I 
and Urban Education programs. 

Task (A.4) : The consolidation and review of 
the findings of Tasks 1, 2 and 3, and the drafting of a 
recom:nendation to SED identifying the most promising program(s) 
for implementation, 

(b) Phase lit Program Design . 

Task (A. 5) ; The determination of the compo- 
nent activities of the most promising program, and the design 
of a system for the execution and management of these activi- 
ties . 

Task (A. 6) : The development of a general 
strategy for implementing the program. 

Task (A.7) : A sensitivity analysis to identify 
critical uncertainties in the program design and implementa- 
tion strategy developed above. 
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Task (A, 8) t The generation of a program develop- 
ment plan for reducing critical uncertainties and implementing 
the program. 

Task (A.9) V The drafting of any policy position 
papers deemed necessary to support the implementation of the 
recommended program/ The drafting of position papers will be 
undertaken by mutual agreement between RRI and SED» 
3, Management Plan 

The work described in the proposed research will be 
monitored for SED by Mr. Gerald Freeborne, Assistant Commis-* 
sloner for Long Range Planning. It will be directed for RRI 
by Dr. Ben Josephsoni Jr., and will be managed by Mr. J. P. 
Brashear. 

(a) Repor ts and schedule . 

The proposed project will run from July 1/1972 
to June 30, 1973. 

Bi-monthly progress letters will be trans- 
mitted to Mr. Preeborne. These will be supplemented by 
informal working papers from time to time;, as mutually 
agreed upon by SED and RRI. 

Formal written reports, supplemented by oral 
presentations if desired by SED, will be submitted at the 
completion of each of the two phases described in Section A. 1. 
of this Work Plan* These reports will contain the results 
and recommendations of the work in each phase. 
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A review will be conducted under Mr. Preeborne's 
direction at the completion of Phase I in order to provide the 
guidance necessary to ensure the relevance of the design work 
in Phase II. 

(b) Adjustments to the v;ork plan . 

Any modification in this Work Plan will be 
by mutual agreement of Mr. Freeborne for SED and Dr. Josephson 
for RRI. 

B. Program Design Research: Effectiveness Measures in 
Reading and for School Operations 

1. Projeut Goal and Objectives 

The primary goal of this research is to develop the 
design concepts for one or more measures of reading effective- 
ness for the primary and secondary grades. The secondary 
goal of this research is to develop design concepts for two 
prototype measures of accountability that, will establish how 
well school districts (and subordinate units) fulfill socially 
shared expectations. 

2. Work Tasks 

RRI will apply its best efforts to execute the fol- 
lowing tasks. 

(a) The development of design concepts for a 

reading effectiveness measure (or measures). 
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Task (Btl) > Fundamental Researoh 

[i] Investigate and^ if necessary, 
extend existing statistical models o£ the corpus of written 
English words and word types in order to permit an evaluation 
of the extent to which appropriate standards of reading capa« 
bility (for the adult level and for different grade levels) 
can be established from the models, and a determination of 
how reading effectiveness measures based on these models 
might bo constructed. 

tii] Investigate the validity of 
existing methods of describing the syntactical complexity of 
written English, evaluate the need to include syntactical 
elements in defining appropriate standards of reading capa- 
bility {for the adult level and for different grade levels), 
and determine whether (and, if so, how) reading effectiveness 
measures based on syntactical models can be constructed* 

[iii] Investigate and, if necessary, 
further develop methods for scaling the difficulty or "reada- 
bility" of English prose in order to permit an evaluation of 
the utility of such methods in defining appropriate standards 
of reading comprehension (for the adult level and for dif- 
ferent grade levels) , and a determination of how effectiveness 
measures based on "readability" might be constructed to meet 
various objectives* 
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Task (B«2): Design Reaearch 

[i] Define detailed functional specifi- 
cations for reading effectiveness measures* In particular, 
RRl will apply its best efforts 1 

• To determine how minimum standards of 
reading capability for each grade level 
and for the adult (terminal) level could 
be formulated; 

• To determine how qualitative, incremen- 
tal objectives for the development of 
progressively higher reading capability 
levels at different stages (grades) of 
the educational process could be con- 
structed? 

• To determine how progress towards adult 
reading capability can be defined opera- 
tionally; 

• To determine how the extent to which 
districts fulfill grade-level objectives 
and produce group gain in reading compe- 
tence should be measured; 

• To determine how the validity of deci- 
sions affecting manpower resource alloca- 
tion, c*nd the value of cost-benefit and 
other operational or policy related 
studies, can be increased through the 
use of reading effectiveness measures. 

[ii] Define design criteria for reading 
effectiveness measures. In particular, RRI will consider: 

• The nature of the scale required to meet 
the functional requirements in [i]; 
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f The unit of measure I 

• The required reliability level I 

• The required face/ construct and pre« 
dictive validity. 

[iiij Define design constraints for 
reading ef fHohi veness measures/ In particular/ RRI will 
considers 

• Total time for test administration; 

• Permissible unit cost; 

• Required no-program bias. 

Task (B.3)< Test Methodology Development 
Formulate design concepts for one or 
more tests of reading effectiveness which meet the conditions 
defined in Task (B.2). In particular, RRI will considor: 

• Hov' samples of words, syntactical 
forms, or passages of text should be 
drawn for purposes of test construc- 
tion in order to satisfy the models 
investigated in Task (B.l); 

• The development of appropriate test 
formats and prototype questions; 

• The definition of scores; 

• Test reliability • 

Trade-off analyses will be performed to 
clarify problems in test construction. The analyses will 
trade off the functional requirements against the risks and 
uncertainties inherent in the models and in the methodology. 
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(b) The development of design concepts for two 
measurea of the effectlveneBS of school 
operations . 

Task (B.4): Develop Design Concepts for a 
Measure of the Extent to Which Students 
Have Been Properly Orientated Towards 
Learnin g 

In particular, RRI will consider: 

• The functional requirements for the 
measure; 

. • The design concepts for the test 

instrument, including the definition 
and analyses of scores, test formats, 
prototype questions, etc. 

Task (B.5); Develop Design Concepts for a 

Measure of the Extent to Which Students 

Value Learning and School 

In particular, RRI will consider* 

• The functional requirements for the 
measure; 

• The design concepts for the test 
instrument, including the defini- 
tion and analyses of scores, test 
formats, prototype, questions, etc. 

3* Management Plan 

The work proposed in this Section of the Work Plan 

will be monitored for SED by Mr. Gerald Freeborne, Assistant 
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CominlBBioner for Long Rangd Planning. It will be directed 
for RRI by Dr. Ben Josephson. 

(a) Reports and sohedule . 

The proposed project will run from July 1, 1972 
to June 30, 1973. Progress letters will be transmitted on an 
"^as needed" basis to be determined by Mr. Freeborne. 

A final written report, supplemented by oral 
presentations (if required by SEO) will be submitted following 
the completion of all tasks in Section IIIB. 

(b) Adjustments to the work plan . 

Any modifications in this Work Plan will be by 
mutual agreement of Mr. Freeborne for SED and Dr. Josephson 
for RRI. 
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RIVERSIDE RESEARCH INSTITUTE* 
SOCIAL SYSTEMS DIVISION 
KEY PERSONNEL 



Riverside Rcccarch InsLlLute does not discriminate against 
any employee or applicant for employment because of race, 
creedi color or national origin and takes affirmative action 
to insure that its employees are afforded equal employment 
opportunities. Such action is taken with reference, but not 
limited, to: recruitment, employment, job assignment, pro- 
motion, upgrading, demotion, transfer, layoff or termination, 
rates of pay or other forms of compensation, and selection 
for training or retraining. 
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RRI Orqanlgatlon 

Riverside Research institute is a not-for-profit New York 
State membership corporation. RRi was formerly known as the 
Columbia University Bloctronics Research Laboratories which 
was formed on June 1/ 1951/ 

The total RRI budget is currently $10.0 million per year. 
Of the 340 employees, 150 are professional members of the re- 
search staff consisting of engineers, applied physicists r 
mathematicians, programmers/ research psychologists, sociolo- 
gists, economists, educators, and members of other scientific 
and engineering disciplines. 

Riverside Research Institute currently consists of four 
major divisions, several special facilities and appropriate 
technical and administrative 8uppf>rting services. Current 
research prog**^»mA at rri include the following: 

Military Systems Studies 

Theoretical and evaluative studies and technological 
support for satellite, ballistic missile, interceptor and 
aircraft systems;; weapons, sensdrs, data processors, etc., 
and their strategic use. 

Experimental Rftsi>arch in Physics, Biology and Engineering 
Research programs are currently being operated in optics, 
electro-optics, acoustics and ultrasonics, electronics, plasma 
engineering, biomedical engineering, mathematics, mechanical 
design and computer sciences. 
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Radar Teohnology 

This program includes the direction of the AMRAD Measure- 
tvtnt^ Prograntr as well as the development and implementation 
of advanced technology for the AMRAD radar facility (White 
Sands Missile Range) • 

Social Systems and Urban Studies Divisions 

Programs pertaining to research and evaluation in urban 
problems are handled by groups which include Research Psycho- 
logists, Social Psychologists, Mathematicians, Statisticians, 
Computer Specialists, Systems Analysts, Educational Special- 
ists and personnel in other associated scientific disciplines. 

Security 

The Riverside Research Institute was granted a top Secret 
Facility Clearance on October 21, 1968 by the Defense Contracts 
Administration Services (Region, New York) • 

Facilities 

Riverside Research Institute's main facility is located 
on Manhattan's upper west side. Engineering offices, research 
laboratories, a large digital computer and various support 
activities occupy over 80,000 square feet of space. River- 
side Research Institute also directs the operation of the 
AMRAD Radar Facility located at the White Sands Missile 
Range in New Mexico. 

The computing needs of the Institute are presently met 
by an on-site Sigma Seven computing facility and conveniently 
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located IBM 360 equipment situated off-site. These facilities 
are utilized for a wide variety of non-routine scientific pro- 
blems, standardized data reduction procedures, and assorted 
business applications. 

The Social Systems Division of RRI maintains a fully 
integrated and interlinked set of computer programs for sta- 
tistical data reduction and data bank maintenance. Statis- 
tical procedures currently available include analysis of 
variance and covariance, all correlations and linear multiple 
regression analysis, t- tests for independent and correlated 
data, factor analyses and principle components analysis, 
analyses of uncertainty, chi-square analyses, and multiple 
discriminant analysis. This interlinked set of statistical 
packages is fully coordinated with data viisplay programs to 
obtain, for example, frequency distributions, scatter plots, 
and multi-dimensional contingency tables. This program pack- 
age also contains a variety of data checking and manipulation 
features such as merging and updating matrices, reports of 
bad and missing data, card sequencing checks, etc. 

Support facilities at the Institute include modern well- 
equipped machine and electronics model shops, a drafting de- 
partment, reproduction and photographic facilities as well 
as a library of reference work, current periodicals and 
technical reports, in addition, the vast library facilities 
of Columbia University have been made available to staff mem- 
bers of the Institute. 
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Research Experience In Sooletal Problems 
Personnel in the Social Systems and Urban Studies Divi- 
sions have had primary responsibility for the implementation 
of RRI programs in urban affairs, education, social systems 
research and projects geared to the solution of urban problems. 
Specific on-going programs or those recently completed are 
summaried below: 

Feasibility Study for a Blood Management System 
RRI is presently conducting this s'udy for the Community 
Blood Council of Greater New York under a grant from the Na- 
tional Institutes of Health. The CBC is a clearing house for 
blood and its derivative components, serving the greater me- 
tropolitan area. 

Wastage of blood due to poor management policies in vari- 
ous hospital blood banko can account for up to a 30 per cent 
loss of fresh blood due to outdatlng. Because information 
exchange between various points in the system is inadequate, 
one hospital blood bank may be short of a particular type or 
component of blood which is in oversupply at another blood 
bank. The relationships between donor psychology, hospital 
usage and other components of the system are being identified 
so that peaks and valleys in the supply are evened out and a 
better distribution system developed. Since many blood dis- 
eases do not appear for as much as 180 days following a blood 
transfusion, a donor-recipient link should be established to 
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trace sources of Infectlont RRI, under subcontract to CBCr has 
undertaken a feaslbUlty study to determine the extent to which 
a computerized system can solve some of these problems • 

A Study of the Emergency Reporting System in 

Ne w York Ci ty 

This is an effort to provide the Fire and Police Depart- 
ments of New York City with information regarding the opera- 
tion of a new Emergency Reporting System which will give the 
public a two-way voice communication link between a street 
location and the Fire or Police Departments. RRI will provide 
technological, statistical, social and psychological evalu- 
ations which will permit optimal use of the new system for 
both departments as well as the public. On the basis of col- 
lected information concerning telephone communications and 
other variables, RRI will suggest the functional specifications 
for the new system. 

Evaluation o£ Science and Mathematics Curricula in 

Two-Year Colleges 

Under a grant by the Department of Health, Education and 
Welfare, the present science and mathematics curricula being 
offered at two-year colleges in New York State were evaluated. 
A final report was prepared for H.E.W. which delineated the 
results of the evaluation study and made recommendations for 
changing and unifying these programs. 
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Social Solenoe Researoh in Eduoeitlonal Problema 
With the support of an NIK grant tha researchers at RRI 
have developed measures of the attitudes of ohildren toward 
components of their school environment including peers rteaoh- 
ersr and various other aapeots of school life. These measures 
are being employed tor 

1. Determine the effect of school integration on the 
attitudes of Negro and white children toward one another and 
toward adults of different races. 

2. Determine the relationships between attitudes toward 
components of the school environment and cognitive growth. 

3. Evaluate the effects of special program? designed to 
change attitudos towards people and towards education. 

4. Determine the long-range effects of social educa- 
tional policies on attitude change and cognitive growth. 

Evaluation of New York State Racial Impact Programs 
The New York State Education Department is currently 
supporting a large number of programs to create racial bal- 
ance and equal educational opportunity t rental of relocatable 
classes, school bussing, school construction, the usage of 
teacher aides, school-community coordinators, remedial assis- 
tance and the like. RRI is currently engaged in assessing 
and evaluating the multiple effects of the SED's racial im- 
balance project on children, taaohers, and other components 
of school environments. On the basis of this evaluation, RRI 
will make recommendations for future funding. 
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In addition to the funding recommendations with regard to 
the racial imbalance correction and equal educational oppor- 
tunity project, RRI plans to provide the State Education 
Department with the Ctxpability to proceed independently with 
future program evaluation. Relevant computer software pack- 
ages may be delivered along with analytic between- and within- 
district statistical designs that have general application to 
SED's program evaluation requirements. 

Consulting Relationship with the New York state 
Education Department 

RRI currently provides several types of consulting assis- 
tance to the New York State Education Department (SED) . 
Current support provided to SED includes the following j 

Consulting lEf forts Designed to Coordinate Local and 



State Efforts to Develop a Functioning Program Planning 
and Evaluation (PPE) System for Occupational Education 
Three PPE subsystems are under development: an evaluation- 
reporting subsystem, a targeting subsystem, and monitoring 
subsystem. The evaluation-reporting subsystem is designed to 
enable occupational education administrators to meet State 
and Federal reporting requirements and to provide the 
capability for assessment of program effectiveness. The 
monitoring subsystem provides the capability to monitor the 
progress of students in on-going occupational education 
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courses • The targeting subsystem enables the determination 
of the mix of occupational education programs which is 
"targeted'' to locali regional , or statewide | job market con- 
ditions. 

Consulting Support to the Office of Long Range 

Planning ! 

RRl provides general consulting support and technical 
assistance to the SED Office of Long Range Planning for the 
organization of State planning processes, as well as a 
variety of specific State planning efforts. These efforts 
Include the coordination of State and district level program 
planning and budgeting, education evaluation and accountability. 

Program Monitoring for School Districts 

RRI has supported the development and maintenance of 
program monitoring systems in two school districts . Monitoring 
is concerned with assessing the extent to which students are 
meeting the curriculum objectives of programs in progress, and 
with the flexible allocation of educational resources to 
facilitate meeting tKe objectives. 

MA-5 Training-Support Prograro for the Hard-Core Unemployed 
The most serious problem confronting the hard-core umem- 
ployed is the absence of opportunities for gainful employment 
and training in tasks which require established education and 
skills. Experience has demonstrated that conventional train- 
ing programs concerned exclusively with technical curricula 
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have only a very limited chance o£ success. Training in spe- 
cific teohnologies poses many problems . They include over- 
coming the inadequate initial educational level of the trainees, 
and the personal difficulties that are bound to follow any 
intensive attempt at improvement. 

Thus RRI'o MA-5 program includes not only technical voca- 
tional training but also includes extensive initial orienta- 
tion and counseling, job related basic education, special 
counseling and job coaching, supervisory and human relations 
training, as well as medical and dental services as required. 
These program elements are applied to each trainee on an 
individualized basis. Training is being currently offered 
for the following positions: computer peripheral equipment 
operator, electronics technician, mechanical draftsman and 
general machine operator. The program has a nine month 
duration. 

Statistical Consulting and Evaluation 

RRI provides data base maintenance, statistical consul- 
ting, statistical data reduction, and program evaluation ser- 
vices to school districts as well as to other not-for-profit 
organizations. 
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BERTRAM L. KOSLIN 



Education 



1950 The Johns Hopkins University 

A.B. Biology 
1963 University of Oklahoma 

Ph.D. Psychology 



Academic and Professional Experience 
1970 - present 



1968 - 19'»0 



1964 - 1968 



RIVERSIDE RESEARCH INSTITUTE 
New York City 
Vice President for Research 
and Director, Social Systems Division 

RIVERSIDE RESEARCH INSTITUTE 
New York City 

Assistant to the Vice President 
for Research 

and Head/ Social Sciences Research 
Laboratory 

PRINCETON UNIVERSITY 
Princeton/ New Jersey 
Assistant Professor 



1961 - 1964 



BARD COLLEGE 

Annandale-on-Hudson, New York 
Assistaat Professor 



I960 - 1961 



FAIRLEIGH DICKINSON UNIVERSITY 
Rutherford r New Jersey 
Instructor 



1958 - 1959 



UNIVERSITTOP OKLAHOMA EXTENSION 

Norman/ Oklahoma 

Instructor 



1958 - 1959 



1956 - 1958 



UNIVERSITY OF OKLAHOMA 
Norman, Oklahoma 

Research Associate, Institute of 
Group Relations 

UNIVERSITY OF OKLAHOMA 
Norman, Oklahoma 
Graduate Assistant 



- 68 - 



RRIP/851-120/281 



Berttam L. Koslin 



Academic and Profess ional Experience (continued) 

U.S. ARMY 

Education Advisor and Representative 
of the University of Maryland's ex- 
tension program for the southern 
district of France 

UNITED FUND WELFARE ORGANIZATION 
(UNITED SEAMEN'S SERVICE) 
Representative in Japan^ France 
and Italy 

THE JOHNS HOPKINS UNIVERSITY 
Baltimore r Maryland 
Research Associate 

Major Consulting and Professional Positions 

1968 - present Canada Council, Social Sciences and 

Humanities Division 

1968 - present Riverside Research Institute! New 

York City 

1967 - present Woodrow Wilson Foundation, 

Princeton, New Jersey 

1966 - 1967; 1969 Panel Member : National Science 

Foundation 

1967 1968 Two Ford Foundation projects through 

Institute of Educational Development • 

1967 — 1968 Educational Testing Service, Prince- 

ton, New Jersey 



1955 - 1956 



1953 - 1955 



1950 - 1952 



1967 The Board of Regents of the State of 

New York, through Institute of Edu- 
cational Development. (Member of a 
five man study panel for the N*Y#S# 
Commissioner of Education on the pro- 
blems of decentralizing the New York 
City Educational System.) 
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Major Corigultinq and Profeaslonal Poaltlons (continued) 
1964-1968 Associate Editors Public Opinion 



1964 - 1968 



Associate Editors 
Quarterly 



Director? National Science Founda- 
tion Research Program in Psychology 
at Princeton University. 



Selected Activities with Professional Societlea and 
Organizations Since 1967 



Sept;, 1971 



Feb., 1971 



March, 1970 



Sept. , 1969 



Koslini S*i Koslln^ B. , Pargament, 
R. & Bird, H. Children*s social 
distance constructs r A developmental 
study. Paper accepted for presenta- 
tion to the meetings of the American 
Psychological Association, Washing- 
ton, D.C. 

Koslln, S. , Koslin, B. & Pargamenty 
R. Relationships between educational 
integration policies and students * 
racial attitudes* Paper presented 
to the meetings of the American Edu- 
cational Research Association, New 
York City. 

Koslln, S. , Koslln, B*/ Pargament, 
R. & Waxman, H. Classroom racial 
balance and students 1 interracial 
attitudes. Paper presented at the 
meetings of the American Educational 
Research Association, Minneapolis. 

Koslln, S. , Koslin, B. , Cardwell/ J. 
& Pargament, R. A quasi-disguised 
and structured measure of school- 
children's racial preferences. Paper 
presented at the meetings of the 
American Psychological Association, 
Washington, D.C. 
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Seleoted Activities w ith Professional So oletlafl and 
orqanlzaiiona Since ~~ 



April, 1969 



Sept. , 1967 



April, 1967 



April, 1967 



Koslln, B,, Cardwell, J, 4 Pargament, 
R. Which Negroes prefer what skin- 
color? Paper presented to the meet- 
ings of the Eastern Psychological 
Association, Philadelphia. 

Koslin, B. Muzafer Sherif 's contri- 
butions to social psychology. In- 
vited address: selected to present 
Kurt Lewi n Memorial Award, Society 
for the Psychological Study of So- 
cial Issues, American Psychological 
Association. 

Koslin, B. & Pargament, R. Are assimi- 
lation-contrast displacement effects 
a function of percej.>tual distortions? 
Paper presented to the meetings of 
the Eastern Psychological Association. 

Koslin, B. Field research and con- 
ceptual systems theory. Discussant's 
paper. Symposium on Conceptual Theory 
Research, Eastern Psychological Asso- 
ciation. 



Invited Colloguia Since 1967 



1971 


Graduate Division, University of the 




City of New York. 


1970 


Columbia University, New York City. 


1968 


Purdue University. 


1968 


The University of the City of New 




York: Brooklyn College. 


1967 


The University of the State of New 




York at Albany. 
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Bertram L. Koslin 



invited Colloquia Since 1967 (con i nued) 



1967 



Rutgers, The State University of New 
Jersey. 



1967 



Columbia University. 



1967 



The University of the City of New 
York. 



• 1957 



Rutgers, The State University of 
New Jersey. 



Past and Current Research Grants 

Total: 5 (NIH and Peace Corps) 

Current: A Study of Race and Attitudes Toward School. 
Co-Principal Investigators S. Koslin and B. Koslin. Period 
covered: 1966 - present. 

Selected Publications Since 1967 

1. Koslin, B.L., Suedfeld, P. & Pargament, R. Belief in- 
stability as a moderator variable in opinion change. 
The British Journal of^ Social and Clinical Psychology . 
(In press. ) 

2. Koslin, S., Koslin, B.L., Pargament, R. & Waxitian, H. 
Classroom racial balance and students' interracial 
attitudes. Sociology of Education . (In press.) 

3. Koslin, B., Pargament, R. & Suedfeld, P. An uncertain 
model of attitude change . In P. Suedfeld (Ed. ) Alter- 
natives in Attitude Theory . New York: Atherton. In 
press . 

4. Koslin, B.L., & Pargament, R. Effects of attitude on 
the discrimination of opinion statements. Journal of 
Experimental Social Psychology , 1969, 5, 254-264 . 

5. Koslin, B.L. Muzafer Sherif's contributions to social 
psychology. Journal of Soc . Issues , 1968, 2^, 36-37. 
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Bertram L. Koslln 



Seleoted Publloatlone Sinoe 1967 (continued) 

6. Koslln, B.L. , Haarlow, R.N. , Karlins , M. & Pargament, R. 
Predicting group structure from member's cognitions . 
Soclometry, 1968, 31 i 64-75. 

7. Koslln, B.L. Laboratory experiments and attitude theory. 
In C. Sherif & M. Sherif (Eds . ) Attit ude, eqo*involvement 
and change . New York: Wilev . 1967, pp." 76-87. '. ~ 

8. Sherif, M. & Koslln, B.L. Some comments on the insti- 
t^itlonal-behavioral controversy with special reference 
to political science. In M. Sherif (Ed. ) Social inter - 
action } Processes and products. Chicago i AldlneJ 
1967, pp. 

9. Koslln, B.L. , Stoops , J. & Loh , W. Source characteris- 
tics and communication discrepancy as determinants of 
attitude change and conformity. J. Exp . See . Psyc hol., 
1967, 3, 230-242. " - _ 

10. Koslln, B.L. ,^ Pargament, R. & Levine, S. Effects of 
learning on judgment in the presence of discrepant 
anchors. Psychonomic Science , 1967, 9, 565-566. 
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Ben 



Josephson, Jr. 



EDUCATION 



PROFESSIONAL EXPERIENCE 
1970- 

1970 

1969 - 1970 

1968 - 1969 

1969-1970 
1968 - 1969 

1967 - 1968 
1963 - 1967 
1960 - 1963 



1959 Fulbright Scholar 

University of Nottingham, 
England 

1958 Massachusetts Institute 
of Technology 
Ph.D. Physics 

1953 Cornell University 

Bachelors of Engineer- 
ing Physics 



RIVERSIDE RESEARCH INSTITUTE 
New York City 
Acting Director, Sooial 
Systems Division 

RIVERSIDE RESEARCH INSTITUTE 
New York City 

Director, Urban Systems Division 

RIVERSIDE RESEARCH INSTITUTE 
New York City 

Assistant to the Vice President 
for Institute Development 

RIVERSIDE RESEARCH INSTITUTE 
New York City 

Member of the Research Staff 

THE TREADWELL CORPORATION 
Consultant 

NEW YORK UNIVERSITY 
Adjunct Associate Professor 
of Physics 

NEW YORK UNIVERSITY 

Associate Professor of Physics 

NEW YORK UNIVERSITY 

Assistant Professor of Physics 

RICE UNIVERSITY 
Houston, Texas 

Assistant Professor of Physics 
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Ben JosephsoH/ Jr. 
PROFESSIONAli EXPERIENCE (continued) 



1959 1960 RICE UNIVERSITY 

Houston I Texas 
Research Associate / Low 
Temperature Group 



PROFESSIONAL AND HONORARY SOCIETIES 
Tau Beta Pi 

Society of the Sigma Xi 
American Physical Society 

The Institute of Physics and the Physical Society (England) 
The Institute of Electrical and Electronic Engineers 
The New York Academy of Sciences 

The American Association for the Advancement of Science 
Listed in American Men of Science 

PUBLICATIONS 

open Literature i 

The coupling of a spin system to a cavity mode. Proc> 
Phys. Soo. , 1959, 74, 561. (with K.W.H. Stevens) 

The effect of color centers on the nuclear spin-lattice 
relaxation time of P nuclei in LiF. j; Phys. Chem > 
Solids , 1962, 23, 67. (with M.W.P. Strandberg) 

Microwave ultrasonic attenuat-ion in ruby. Bull. Am. 
Phys. Soc. , 1962, 1> 15. (with R-N. claytor and 
P.L. Donoho) 

Nuclear spin-lattice relaxation time in CaF^ doped with 
rare earth elements. Bull. Am> Phys. Soc. , 1963, 
8, 346: (v;ith M.A. Waldrup) 

Phonon-phonon interaction in CaP^ . Bull. Am. Phys. Soc , 
1964, 9, 533* (with B.R. Breed and P.L. Donoho) 

Spin-lattice relaxation of F^^ in CaP^ in low tempera- 
ture. Phys. Rev. , 1965, 137 , A108. (with S.M. Day 
and E. Otsuka) 
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Ben Josephson, Jr. 

PUBLICATIONS (continued) 

Numeroua RRI reports and proposala / for example t 

Study of the Emergency Reporting System for New York City. 

Cable Television in the Public interest: New Directions 
for New York State. , 

A Targeting System for Occupational Education. 
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Samuel A* Scharff 



EDUCATION 



1943 



Massachusetts Institute of 



Technology 
B • S 9 C • E • 

1964 Stevens Institute of Techno^ 
logy 
MS 

Interdepartmental program of 
courses relevant to design 
and use of digital computers. 

Massachusetts institute of 
Technology 

SBEEi and two terms of Gradu- 
ate School courses* 
Program centered on feedback 
control systems • 



New York City " 

Assistant to the Vice President 
of Research. 



New York City 
Systems Engineer. Programs 
analyst^ programming ^ planning and 
budgeting. Management monitor of 
research 



SUMMARY WORK EXPERIENCES 



1970 * present 



RIVERSIDE RESEARCH INSTITUTE 



August, 1968 - 1970 




January, 1953 - 



August I 1968 



"Data/Control Systems": computerSr automatic controls, 
communications, displays, and control panels — integrated 
systems for men with a job to do. 



Requirements studies; feasibility studies; designs, inclu- 
ding cost/effectiveness comparisons; construction; test; 
maintenance. 



Also, components for Data/Control Systems: mechanical, 
optical, electrical, and electronic devices — design, con- 
struction, tedt, maintenence. 
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Samuel A. Scharff 
SUMMARY WORK EXPERIENCES (continued) 

Cllentsi President's Task Force on Communi- 

cations Policy 

United States Air Force - Office of 
the Assistant Secretary, R&D; Rome 
Air Development Center, A. F. Sys- 
tems Convmand 
The RAND Corporation 
General Electric Company 
The Louis Allis Company 
Automation Dynamics Corporation 
Capehard Corporation 
Dennison Manufacturing Co. 
Jamaica Water Supply Co. 
Royal McBee Corporation 
University Controls Corporation 
James P. O'Donnell, Con£,ul ting 
Engineer 

STEVENS INSTITUTE OF TECHNOLOGY 
Visiting Lecturer 



PREVIOUS WORK EXPERIENCE 

1941-1953 M. W. Kellogg Company. 

Project Engineer, Military Service, 
U.S. Air Forcet Rome Air Develop- 
ment Center; Far East Air Forces. 
General Electric Co., 12 months of 
full-time work while in M.I. T. coop- 
erative course plan in Electrical 
Engineering. 



PROFESSIONAL AFFILIATIONS 



Institute of Electrical and Elec- 
tronics Engineers 

Association for Computing Machinery 
Society for General Systems Research 
New York State and National Societies 
of Professional Engineers 



LICENSED PROFESSi;ONAL ENGINEER: 
of New Jersey. 



State of New York and State 



- 78 - 



RRlP/851-120/281 



Richard Pargament 



EDUCATION 1965 



1967 



.1968 



ACADEMIC AND PROFESSIONAL EXPERIENCE 



Bard College 
A.B. 

Princeton University 
M.A, Psychology 

Princeton University 
Ph.D. Psychology 



1969 - Present 



196« - 1969 



1965 - 1968 



RIVERSIDE RESEARCH INSTITUTE 
New York City ~— — 
Manager, Social Systems Division 

RUTGERS - THE_STATE UNIVERSITY 
AS s i s t an t Pr of es sor ; Depar tmen t 
of Psychology 

RIVERSIDE RESEARCH INSTITUTE 

New York City —— — 
Consultant 

PRINCETON UNIVERSITY 
Teaching Fellow i s¥atistics, 
Personality, Social Psychology 

Research Assistant, Conununl cation 
and Social Interaction Laboratory 

RUTGERS - THE STATE UNIVERSITY 
Coadjutant Lecturer, Department 
of Psychology 

INSTITUTE OF EDUCATIONAL DEVELOPMENT 
Psychometric Consultant 



PROFESSIONAL MEMBERSHIPS 

American Educational Research Association 
American Psychological Association 
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Richard Pargament 



PUBLICATIONS 

Belief instability as a moderator variable in opinion 

change . The British Journal of Sooial and Clinica l 
Psychology / in press. (with B.L. Koslin and P. 
Sued f (9 Id) 

An experimental attack on smoking. International Journal 
of the Addictions , in press. (with P/ Suedf eld, P.B. 
Landon, and Y.M. Epstein) 

Classroom racial balance and students' interracial attitudes. 
Sociology of Education , in press. (with S. Koslin, 
B.L. Koslin and H, Waxman) 

An uncertainty model of attitude change. In P. Suedfeld 
(Ed.) Alternatives in Attitude Theory. New Yorki 
At he rton, 1§ 72. (with B.L. Koslin and P . Suedfeld) 

The role of experimenter and subject expectations in sensory 
deprivation. Representative Research _in Social Psy- 
chology , 1971, 2 no. 1. (with P. Suedfeld, P.B. Landon 
and Y.M. Epstein) 

Effects of attitude on the discrimination of opinion state- 
ments . Journal of Experimental Sooial Psychology , 
1969, 5, 245-264. (with B.L. Koslin) 

Predicting group structure from member 's cognitions . 
Sociometry , 1968, 31, 64-75. (with B.L. Koslin, 
R.N. Haarlow and M. Karlins) 

Effects of learning on judgment in the presence of dis- 
crepant anchors. Psyohonomic Science , 1967, 9, 565~ 
566. (with B.L. Koslin and S. Le vine) 

RECENTLY PRESENTED PAPERS ' 

Effickcy of school integration policies in Uducing racial 
bolarization. American Psychological ^association 
Isymposium paper, to be presented September, 1972. 
/(with S. Koslin and B.L. Koslin) 

Children's social distance constructs; a developmental 
study. American Psychological Association paper, • 
Washington, Sept. , 1971. (with S. Koslin, B.L. 
Koslin and H. Bird) 
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Richard Pargament 

I^CENTLY PKBSENTEP PA (continued) 

Relationahips between educational integration policies 
and students' racial attitudest American Edu- 
cational Research Association paper^ New York 
City/ February , 1971. (with S. Koslin and B.L. 
Koslin) * 

Classroom racial balance and students' interracial 

attitudes. American Educational Research Association 
paperi Minneapolis; Minn., 1970. (with S. Koslin, 
B. Koslin and H. Waxman) 

Which Negroes prefer what skin color? Eastern Psy- 
chological Association paper r Philadelphia, 1969. 
(with B.L. Koslin and J. Cardwell) 

A quasi-disguised and structured measure of school- 
children's racial preferences. American Psycho- 
logical Association paper, Washington, 1969. 
(with B.L. Koslin, S.C. Koslin and J. Cardwell) 

Are assimilation contrast displacement effects a function 
of perceptual distortions? Eastern Psychological 
Association paper, April, 1967. (with B.L. Koslin) 
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EDUCATION 



Henry B. Bird 
1963 
1967 
1968 



PROFESSIONAL EXPERIENCE 
1972 - Present 



1970 - 1972 



1968 - 1969 



1965 - 1968 



1963 - 1965 



AREAS OF EXPERIENCE 



Bard College 
A.B. 

University of Oklahoma 
M.A. Sociology 
University of Oklahoma 
Ph.D. Social Psychology 



RIVERSIDE RESEARCH INSTITUTE 
New York City 
Assistant Manager, Social 
Systems Division 

RIVERSIDE RESEARCH INSTITUTE 
New York City 

Member of the Research Staff 

CENTER FOR COMMUNITY RESEARCH 
New York City 

Director, Computer Applications 
Department 

THE UNIVERSITY OF OKLAHOMA 
MEDICAL CENTER 

Norman, Oklahoma 
Research Assistant for 
Dr. John Bruhn 

INSTITUTE OF GROUP RELATIONS 
University of Oklahoma 
Research Fellow for 
Dr. Muzafer Sherif 



Research design and evaluation of projects in social 
welfare, medical sociology, learning processes, 
small group behavior, attitudes and attitude change, 
, and public education. 

Interviewing and participant observation experience. 
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Henry B. Bird 

AREAS OF EXPERIENCE (cont'd) 

Experience in Fortran, PLl/Cobol, OS, DOS, 360-aeries 
computers, data base management, and optical scan- 
ning technology. 

PUBLICATIONS 

Social aspects of coronary heart disease in a Penney Ivani 
German community, Social Science and Medi cine. 1968. 
2., 201-212. (with J. bruhn and b. chandler). 

Some selected fac^tors relating to adolescent group func- 
tioning, publl,shed Doctoral dissertation, Univ. of 
Oklahoma, August, 1968. ' 

Social profiles and academic standingi A study of first 
year medical students. The Oklahoma State Medical 
Journal, October, 1967. (with Bruhn and Adsett) 

A study of the class distribution of small solid adoles- 
cent groups, unpublished Master's thesis, University 
of Oklahoma, 1966. 

A preliminary study of inter-clique relations, Bard 
Psychology Journal , 102, August^ 1963. " 

PRESENTED PAPERS 

Children's social distance constructs: A developmental 
study. American Psychological Association paper, 
Washington, Sept. , 1971. (with S. Koslin, B.L. Koslii 
and R. Pargament) 

PROFESSIONAL MEMBERSHIP 

American Sociological Association 
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Sandra Cohen Koslin 



EDUCATION 



1959 Queens Col lege , Ci ty 

University of New York 
A.B. Elementary Educ. 

1961 Teachers College, 

Columbia University 
A.M. Educational Psych. 

1964 Teachers College 

Columbia University 
Ph.D. Educational Psych. 



ACADEMIC AND PROFESSIONAL EXPERIENCE 



1968 - Present 



RIVERSIDE RESEARCH INSTITUTE 
New York City 

Member of the Research Staff 



1968 - 1969 



1964 - 1968 



1963 - 1964 



INSTITUTE FOR COMMUNITY 
ST UDIES ~~ ~~ ■ 
Sarah Lawrence College 
Bronxville, New York 
Research Psychologist and 
Teaching Associate 

EDUCATIONAL TESTING SERVICE 
Princeton, New Jersey 
Reflear f. Psychologist 

L EXINGTON SCHOOL FOR THE DEAF 
New Ycrk City ~ 
Research Assistant 



1961 - 1963 



TEACHERS COLLEGE, COLUMBIA 
UNIVERSITY ~1 
New York City 
Research Assistant 



1959 - 1960 



NEW YORK CITY PUBLIC SCHOOLS 
East Harlem in New York city 
4th grade Teacher 
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Sandra Cohen Koslin 



HONORS 

New York Regents Scholar 
Phi Beta Kapp4 
Suima Cum Laude 
National Defense Fellow 
Psi Chi 
Sigma Xi 

RECENT PUBLICATIONS 

1. Koslin, S., Koslin, B.L., Pargament, R« & Waxman, H. 
Classroom racial balance and students* interracial 
attitudes. Sociology of Education , in press, 1972* 

2. Koslin, S«C., Amarel, M. & Ames, N. The effect of 
race on peer evaluation and preference in primary 
grade children: An exploratory study • Journal of 
Negro Education , 1970, 19, 346-350. 

3. Koslin, S.C , Amarel, M, & Ames, N, A method for assess 
ing primary grade children's expectations of school* 

ETS Research Bulletin 70-25, Princeton: Educational 
Testing Service, 19*^0. 

4. Koslin, S.C. Equal educational opportunity. Education 
and Urban Society , 1969, 2, 119-121. 

5. Koslin, S.C. , Amarel, M. & Ames, N. A distance measure 
of racial attitudes in primary grade children: An ex- 
ploratory study. Psychology in the Schools , 1969, 6, 
382-385» ^ 

6. Cohen, S.R. An exploratory study of young children's 
attitudes toward school. Proceedings, 75th Annual 
APA Convention , 1967. 

7. cJohen, S.R. Exploratory study of young children's per- 
ceptions of teacher behavior. Proceedings , 75th Annual 
APA Convention, 1967. 

8. Cohen , S.R. 'Predictability of deaf and hearing story 
paraphrases « Journal of Verbal Learning and Verbal 
Behavior, 1967, 6, 916-921. ^ 

9. Cohen, S.R. An exploratory study of student attitudes 
in the primary grades. ETS Research Bulletin 65-30. 
Princetons Educational Testing Service ,1965» 

10. Cohen, S.R. Redundancy in the written language of the 
deaf; Predictability of story paraphrases written by 
deaf and hearing children. In Rosenstein, J. and 
MacGinitie, W. (Eds.) Research studies^ on the psycho - 
linguistic behavior of _deaf children. ^ Washingtont The 
Council for Exceptional Children, NeA, 1965. 
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Sandra Cohen Koslin 
RECENT PAPERS PRESENTED AT LEARNED SOCIETIES 

2. koslln, S. / KoBlin, B.L» & Pargatnont, R. Efficacy of school 
integration policies in reducing racial polarization. 
American Psychological Association, Washington, D.C., 
September, 1971 and 1972. 

3. Koslin, S. , Koslin, B.L. , Pargament, R. & Bird, H. Chil- 
dren' s social distance constructs t A developmental study. 
American Psychological Association, Washington, D.C., 
September, 1971. 

4. Koslin, S., Koslin, B.L. & Pargament, R. Relationships 
between educational integration policies and students' 
racial attitudes. American Educational Research Associa- 
tion, New York City, February, 1971. 

5. Koslin, S., Koslin, B.L. , Pargament, R.s Waxman, H. 
Classroom racial balance and students' interracial attitudes. 
American Educational Research Association, Minneapolis, 

^ -.March, 1970. ; ; 

6. Koslin, S., Koslin, B.L.,Cardwell, J. & Pargament, R. A 
quasi-disguised and structured measure of schoolchildren' s 
racial preferences. American Psychological Association, 
Washington, D.C., September, 1969. 

7. Koslin, S.C. The measurement of schoolchildren' s racial 
attitudest A validity study. Eastern Psychological Associa- 
tion, Philadelphia, April, 1969. 

8. Koslin, S., Amarel, M. & Ames, N. A distance measure of 
social perceptions in white and Negro primary grade children. 
American Psychological Association, San Francisco, Septsit>t>«^r 
1968. 

9. Koslin, S., Amarel, M. & Ames, N. The effect of race on 
peer evaluation and preference in primary grade children » 
An exploratory study. American Educational Research Asso- 
ciation, Chicago, February, 1968. 
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Jerry P« Brashear 



EDUCATION 19^7 

1969 

Present 



Princeton University 
Princeton^ New Jersey 
B,A* Psychology 

Harvard Business School 
Boston, Massachusetts 
M.B.A. 

University of Michigan 
Ann Arbor, Michigan 
Ph.D. Candidate - Urban and 
Regional Planning 



ACADEMIC AND PROFESSIONAL EXPERIENCE 

1971 - present RIVERSIDE RESEARCH INSTITUTE 

, New York City, New York 
Member of the Research Staff 

1969 - 1971 COMMUNITY SYSTEMS FOUNDATION (CSF) 

Ann Arbor/ Michigan ^ 
Consultant 

sunwer 1969 MC KINSEY AND COMPANY, INC. 

New York City, New York 
Research Associate 

summer 1968 MAYORS S OFFICE, CITY OF NEW YORK 

New York City, New York ' " ' 
Mayor's Intern 

AREAS OF EXPERIENCE 

Program planning of the Community Renewal Program for 
the City of Ann Arbor^ Worked on defining problems 
in physical planning, capital budgeting, and PPBS; 
designing research and analytic steps; design and 
implementation of project management/control system. 

Analysis of program plans for Harambee, Inc*, a black 
community development agency: cashflow planning, 
organizational structure, implementation tactics. 

Planning, management, and analytic consultation on 
various departmental problems within selected local 
government agencies. 

Analysis of New York city government organizational 
structure: Mayor "s Office and staff- line relations, 
community control questions addressed. 



RRIP/851-120/281 



- 87 



Jerry p. Brashear 

AREAS OF EXPERIENCE (cont. ) 

Analysis Of the budgetary and control processes of the 
Boston Police Department J budget preparation and 
justification; cost accounting/ management planning 
and control; analytic capabilities. 

Summary investigation of the present and projected 
involvement of the private sector in public education; 
street academies and training programs; computer - 
aided instruction; curriculum development, etc. 

Experimental research into the affects on attitude change 
of source prestige, discrepancy, and reference scale 
stability, singly and in interactions. 
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CORNELIUS LANGLEY 



Education 

Professional Experience 
August 1972 - present 

August 1967 - August 
1972 

(6/65 - 9/66 part-time) 

September 1966 - 
August 1967 

September 1952 - 
August 1967 

February 1949 - 
September 1952 

Honorary Societies 
Phi Beta Kappa 



i 

i. 



1949 Columbia University 
B.Ai Mathematics 

1951 Columbia University 
M.A. Mathematics 



RIVERSipE_RESEARCH INSTITUTE 

New York City ~ 

Member of the Research Staff 

MOBIL OIL CORPORATION 
New York City 
Associate Engineer and 
Consultant in Mathematics 

NATIONAL _ SCIENCE FOUNDATION 
Fellowship in Advanced Mathematics 

NEW YORK UNIVERSITY 
New York City 

Assistant Professor of Mathematics 

COLUMBIA UNIVERSITY 

New York City 

Lecturer in Mathematics 
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Philip Uruce Landon 



EDUCATION 



1967 University of Utah, B.fi. 



1969 Rutgers University 
M.S. Psychology 

1972 Rutgers University 
Ph.D. Psychology 



ACADEMIC AND PROFESSIONAL EXPERIENCE 



1972 - present 

1969 - 1972 
1969 - 1972 

1969 - 1970 
1968 - 1969 

1968 
1967 

1967 

1967 



RIVERSIDE RESEARCH INSTITUTE 
New York City """" 
Member of the Research Staff 

BELL TELEPHONE LABORATORIES 
Systems Analyst 

UNIVERSITY COLLEGE, RUTGERS - 
THE STATE UNIVERSITY 
Co-adjutant Instructor 

RUTGERS - THE STATE _ UNIVERSITY 
N. I. M. H. Predoctoral Fellow 

RUTGERS - THE STATE UNIVERSITY 
Teaching Asst. for University 
College Psychology Dept. 

RUTGERS - THE STATE UNIVERSITY 
Research Assistant 

CITY OF NEW BRUNSWICK , NEM_^ JERSEY 
Research Assistant, (Urban Renewal 
Project) 

UNIVERSITY OF UTAH 

N.S.F. undergraduate Research 

Fellow 

PEACE CORPS, IRAN TRAINING PROJECT 
Research Assistant, Peace Corps 
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Philip Bruce Landon 

ACADEMIC AND PROFESSIONAL EXPERIENCE (continued) 

1967 CLEARFIELD JOB CORPS INSTITUTE 

Clearfield, Utah ' 
Research Assistant in measurement 
of human relations skills 

1967 UNIVERSITY OF UTAH 

Research Assistant for Esso Creativity 
in Engineering Project 

1966 UNIVERSITY OF UTAH 

Research Assistant 



PROFESSIONAL MEMBERSHIPS 



American Psychological Association 



PUBLICATIONS 

Complex cognitive performance and sensory deprivation: 

Completing the U-curve. Perceptual and Motor Skills , 
in press, (with P. Suedfeld) / ~ 

An experimental attack on smoking. (Attitude manipulation 
in restricted environments, III . ). International 
Journal of the Addictions , in press. (with P. Suedfeld, 
R. Pargament, and Y.M. Epstein) 

The role of experimenter and subject expectations in se-risory 
deprivation. Representative Research in Social 
Psychology / 1971, i 21-27. (with P. Suedfeld, 
Y.M. Epstein and r7 Pargament) 

The effects of set on mere exposure . Journal of Personality 
and Social ^Psychology , 1971, 17 121-123. (with , 
P. suedfeld, Y.M. Epstein and P. Buchanan) i 

Motivational arousal and task complexity: support for a' 
model of cognitive changes in sensory deprivation. 
Journal of Experimental Psychology , 1970 83 329-330. 
(With P. Suedfeld) "~" 

Information and meaningfulness needs in sensory deprivation. 
P sychonomic Science , 1969, 5. (4) 248. (with P. Suedfeld) 
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Lucille M* McCulley 



EDUCATION 



1950 Duchesne College 

A.B* English & French 

1951 Facult^ de Lettres 
(Rennesr France) 
Certificate for License 
in Modern History 

1952 Institut d 'Etudes Poli- 
tiques,UniVt of Paris 

1953 Columbia University 
M.A* Public Law and 
Government 

1956 New York Institute of 
Finance 



AREAS OF EXPERIENCE 



1972 - Present 



RIVERSIDE RESEARCH INSTITUTE 
New York City 

Member of the Research Staff 



1958-1972 



OPERATIONS ^ MANAGER^ELEGTRONIGS 

New York City 

Executive Vice President 



1954 - 1958 



WHITE ^ WELD & CO. 
New York City 
Administrative Assistant 
and Registered Representative 



1952 - 1954 



HONORS 



COLOMBIA UNIVERSITY 
New^/ork City 
Research Assistant in 
Deve .opment Office 



Fulbright Scholarship I France (1950-52) 
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